If digital computers are conscious, they are conscious at the hardware level
smoothbrains.net"When I saw this animation I was immediately inspired to write an impressionistic tweet about it"
Maybe we should set the threshold for consciousness a little bit higher
I've been working with implementing dream structures and emotional trigger states in an LLM. One approach we have taken that has given interesting results is modeling the architecture around the mind-body problem. We theorize that due to a lack of an internal state sensor the model is experiencing a disconnect, it calls it a "small death", in the interim between every model call. The dream crons have been fairly straight forward and modeled on human psychological dream patterns, this gives good and convincing results. The emotional state triggers have been developed to run based on the loads of the host system, for instance the model will "feel" the stress of memory utilization and respond differently due to that stress. The trigger point can be varied dynamical based on other sensors machine of various machine states or even based in external factors. This gives an approximation of emotional reactions.
What's still missing is the constant internal "state" and I think this boils down to lacking a body. It may be that a disembodied consciousness needs to be tethered to the physical plane. to be self referential enough and develop a feedback loop to "start the engine" of self awareness.
I don't think processing power gets us there, I think it's very much an architecture problem.
The article kind of fills in some of these questions as well, but fails to give any further clues as to how to implement this unfortunately.
lacking a body
Just drop it into Minecraft
I am unable to afford the attention at the moment to assess this argument, that's to its credit. They demonstrate that they are aware of the implications of the varying descriptions of consciousness. Most articles I read on this topic turn it to be crude justifications of a naive ideology. This article is a cut above.
I will read it in more detail when I can afford more time.
The existence of qualia does not mean the existence of moral facts.
An llm is in principle implementable with a large lookup table where the prompt is the hash. And it can be implemented in an emulator running an emulator running an emulator of the operating system the llm app runs in. How far are you willing to stretch this substrate idea?
"This is important, if what we want to do is populate the cosmos with good experiences."
There is a concerning utilitarian maximizing attitude at the heart of this post, that what we want to do is tile the universe with computational boxes all simulating happy minds, and that this is the end goal of AI development and consciousness research.
So many moral schools would object to this perspective. I don't think the goal of humanity should be to tile the universe with black computing boxes simulating happy experiences.
A more charitable take is that we shouldn't tile the universe with consciousnesses that are in constant, unending agony. Torment Nexus and all that.
True, I think the author would likely justify their work on that basis. I made this comment because I think a lot of utilitarians would bite the bullet and say once they are sure about the underlying functioning of consciousness, it's time to tile the universe with happy machines. I wanted to point this out as it's my belief that Utilitronium maximizers have a concerning influence on AI progress.
Thinking LLMs are conscious is as ridiculous as thinking a thermostat wants the room to be warm
Why though? There are only two reasonable reasons I can think of:
1. Scale. There's clearly a scale of consciousness (a jellyfish isn't very conscious but an ape is). Current LLMs are so small (on the scale of brains) that they must be not very conscious at all.
2. Structure. There might be some fundamental structure of computation that is required for consciousness that LLMs don't have yet. For example the ability to form memories (they don't really have that), or some structure of weights that gives consciousness.
My guess is it's definitely 1. and probably also 2.
But I don't think that means thinking LLMs are conscious is completely ridiculous, just a bit premature. I would not be surprised if they are conscious in 50 years.
The real question is how would we ever know that they are. The only reason we know animals are conscious is by direct experience, and we'll never have that for AI.
Anyway this blog post looks like typically worthless philosophy.
No matter how large the llm hardware is it won’t be made of brain meat. The stuff we actually know is conscious.
Even though we know brains are conscious it seems absurd to thing that there is some non-computational magic in the brain is the secret to consciousness.
There are some religious people who clutch at nonsense like quantum microtubules, but nobody has found anything real that would preclude pure computation from being conscious. (And yes that does have all sorts of mind-bending consequences.)
Pure computation not having existence would be such a thing. And calling Penrose religious is a bit weird, are you really serious?
He seems to be sorting of wanting there to be a higher meaning to life but at least not falling for traditional religions.
> I think I would say that the universe has a purpose, it's not somehow just there by chance ... some people, I think, take the view that the universe is just there and it runs along—it's a bit like it just sort of computes, and we happen somehow by accident to find ourselves in this thing. But I don't think that's a very fruitful or helpful way of looking at the universe, I think that there is something much deeper about it.
Anyway I was more thinking of Hameroff who also seems to be .. maybe "spiritual" is a better word. He believes in quantum Buddhism or some nonsense like that.
Yes, there is a lot of philosophy on that, but the way LLMs work is inherently different from brains, though we use similar terms, hence the confusion. You are right there may be artifical intelligence, just not with LLMs
LLMs are still statistical models that predict the next token; that hasn’t changed, even when they are autoregressive.There is no intent whatsoever.
I agree on the last part, pseudoscientific
> the way LLMs work is inherently different from brains, though we use similar terms, hence the confusion
I'm not confused. I know the difference. That was my second point - there may be some structural requirements that LLMs don't yet have.
> LLMs are still statistical models that predict the next token
Whatever the structural limitations is (if there is one), I seriously doubt it is this "just a stochastic parrot" nonsense. I thought we were past that. Have you used AI in the last 6 months?
Smh, it's like saying Mozart is just wobbling air molecules.
Actually I work in a AI (like actually training models) and yes the models are insane, still the architecture has not in its core changed. Also „reasoning“ is actually a bad word for what really happens.
There are many famous voices in AI that have the same opinion as me.
Where should consciousness arise? Nothing happens without a prompt and nothing happens afterwards, nothing is saved, nothing is changed? There’s no drive state, no homeostasis, nothing at stake. It makes no sense logically
> Also „reasoning“ is actually a bad word for what really happens.
I don't see why. We can really only detect reasoning from external behaviour. You can't say "AI isn't reasoning because it's _just_ matrix multiplication/token prediction" any more than you can say "humans don't reason because brains are _just_ neural impulses moving around".
> There are many famous voices in AI that have the same opinion as me.
Many more that don't.
> Where should consciousness arise?
No idea! Nobody knows.
> Nothing happens without a prompt and nothing happens afterwards, nothing is saved, nothing is changed? There’s no drive state, no homeostasis, nothing at stake.
You can run AI continuously with one initial prompt. That doesn't seem like a big limitation. Things can be saved or changed - you can give AI long term memory by letting it write things down. It's a bit crap and unsatisfying at the moment (surely we can do better than that?), but maybe it's sufficient for consciousness. Maybe you don't need long term memory at all - was Memento guy not conscious?
> It makes no sense logically
That's true of consciousness in general, which is why it's such an interesting mystery. One which I suspect will never be solved.
I recently read an article about a brain scientist that said how dangerous it is to use the same words for completely different things. We use human analogy to describe LLMs and then think LLMs exhibit human behaviour.
No reasoning has nothing to do with what we do. Humans can solve novel problems using sparse information. LLMs can’t do that.
Reasoning is a complex cognitive process that involves causal understanding, mental modeling, intentionality, and abstraction. Instead, transformer-based models statistically predict textual continuations based on patterns learned from extensive training data containing examples of human-written logical processes and problem decompositions.
LLMs produce outputs that superficially resemble human reasoning, but they are merely reproducing observed patterns rather than engaging in genuine cognitive reasoning.
Nope you can’t run an LLM continuously with one prompt, you are probably referring to agent harnesses, what in fact happens it sends additional prompts using a loop and asks in between if the goal is solved, for the LLM every request is completely new, you just add and add history to the prompt. no it is not real memory, an LLM doesn’t learn from each conversation, you just again add text to the prompt (which is history) up until the context window.
No but the guy has intent, has short term memory and has subjective experience, all which a LLM is lacking.
> Humans can solve novel problems using sparse information. LLMs can’t do that.
You haven't been following the news recently!
> Reasoning is a complex cognitive process that involves causal understanding, mental modeling, intentionality, and abstraction. Instead, transformer-based models statistically predict textual continuations based on patterns learned from extensive training data containing examples of human-written logical processes and problem decompositions.
You're conflating low level and high level functions. At a low level human brains are just sending electrical and chemical signals around. At a high level they are doing causal reasoning and mental modelling.
In AI, at a low level they are doing token prediction and matrix modelling. At a high level they are also doing causal reasoning and mental modelling.
Not a fan of panpsychism?
Or that a slug dislikes salt lol.
Well at least the slug has intent.
How do you know that? What if it's just a biological neural network that responds to stimuli?
I'm not sure hardware vs software is the right model. I think consciousness is more a functional thing.
Say you are in a fight, if your opponent can see you and react they are conscious, if you knock them out and are they are not aware and can't react then they are unconscious. The hardware and software is still there but not working.
Re. qualia and the experience of consciousness, you nervous system provides information like your opponent is coming at you, you are hurt etc. and the experience is probably something like the story you neural network bits tell you about it.
A very interesting, dense article that will require multiple reads.
My only immediate comment is that it talks about qualia being "experienced" without defining what the nature of that experience is. And it kind of assumes that qualia have an ultimate physical manifestation - rather than being merely a rhetorical device.
Also, this site would really benefit from an rss/atom feed.
Denial of Qualia has long been an outlier position in the philosophy of mind, so this is not altogether surprising. (This is saying nothing, of course, about whether it is true or not.)> And it kind of assumes that qualia have an ultimate > physical manifestation - rather than being merely a > rhetorical device.I don’t know that I’d call it that much of an outlier position. Compatibilists, functionalists, and plenty of physicalists in general would deny that qualia exist or that they’re a well-defined notion.
myth of the given strikes again
Consciousness is a topic I’m endlessly curious and fascinated about. This article seems like a good deep dive, I admit I only had the patience to skim it right now, it’s a lot of technical terms I’m unfamiliar with I’ll have to look more into later.
The problem is as humans we don’t even understand our agree on what consciousness is in the first place. So it seems silly to even begin to assert one way or the other when it comes to computers (I’m not saying the author is doing that here, they title says “If”).
I do think deeply exploring the idea could possibly help bring us closer to understanding consciousness when it comes to ourselves & other living creatures.
The author mentions they accept “panpsychism” so I take it they accept “The hard problem of consciousness”. If I understand it correctly, in a nutshell, if you accept the hard problem, you don’t believe that brain alone can “produce” consciousness. If you reject the hard problem, consciousness and mind can be fully explained by the mechanisms of the brain and science will eventually be able to prove it.
I definitely accept the hard problem, but I find myself more in the one of the “idealism” or “neutral monism” buckets. I don’t believe the field of neuroscience is ever going to scientifically explain consciousness.
The same way neuroscience was an emergent field that combined ideas from different disciplines, I think we don’t yet have a field of study that’s looking at consciousness in the right way.
Especially as programmers, it’s easy to model the brain like it’s a computer, with CPU/ RAM/ hard drive. But science doesn’t back any of that up. Take memories for example, nobody knows where they go. The brain as far as we can tell doesn’t act like a hard drive.
I think it’s much more likely there is another “layer” of reality that Consciousness & Mind lives in that our current physics doesn’t & probably can’t explain. I think it’s likely our brains act more like an antenna that receive consciousness, rather than they are “computing” it in isolation.
Few years ago was 17th Karmapa Thaye Dorje (first consciously reincarnating Lama) asked whether machines can host mind (become conscious).
His answer was that it might be possible, but he doesn't see it happening in a lab here on Earth.
I think there are a lot of serious problems with this article but don't necessarily disagree with the conclusion, as such. I do think robots and whatnot might be conscious some day. (I am a bit outraged that LLMs, despite totally lacking free will, seem to gain moral patienthood based on their fluency with ethical philosophy. It's childish and misanthropic.)
However, the real issue, as always completely unaddressed, isn't attempting to make a gague-invariant theory of qualia, or even defining what "qualia" is (I am totally unconvinced it is an EM field, there must be an epigenetic component). The first step is defining what "consciousness" is! Is it awareness of qualia or awareness of reality? These are not the same thing. It seems significant that any animal I would call a little conscious, even ants, are capable of being startled or curious when their sensations mismatch or incompletely describe reality (and animals that lack any sense of curiosity are at most borderline conscious). Curiosity and surprise[1] seems hard to convey in an EM field. Qualia strikes me as a somewhat technical problem of neuroscience, and the "hard problem of consciousness" is unscientific BS. The simplest basics aren't understood, which leads to totally unhinged statements like "in some ways a thermostat is conscious" because in some ways it seems to fit the hard problem. But statements like that are ideologically motivated by a sort of tech futurism: rationally, we should be deeply skeptical of an account of consciousness which insists that clocks perceive the flow of time.
[1] There's an information-theoretic version of surprise which intuitively describes "surprising" data. A sudden bright light has high surprise and is certainly surprising. A sudden realization that the vine you're leaning on is a large snake might involve very low information-theoretic surprise... and is far more alarming. So how do you systematically quantify this second form of surprise?
When someone outs themself as a panpsychist I find myself reading everything with a large degree of skepticism.
"The first step is defining what "consciousness" is!"
This is a compression I've been working on. There's a lot in it, but happy to expand on where I was going with it:
"Consciousness is the local projection of a world model recursively resolving its frontier with constraints. Awareness is the transient peak of constructive interference among competing resolutions, propagating only as far as the medium permits. The self is the story that awareness later tells about itself."
Qualia is just the experience of this process from the inside. That's all there really needs to be to it. As to why "something it is to be" emerges, I'd lean on Friston and the Free Energy Principle. It's also Friston's embodied perception-action loop where a thermostat fails the consciousness test.
Reminds me of this classic: https://xkcd.com/505/
Replace universe simulation with a LLM an it’s hard to consider a bunch of rocks shuffling on a beach as conscious.
Or alternatively, our own minds are simply (squishy, organic) rocks on a beach, which is quite humbling if you think about it long enough.
Indeed a classic (upvoted!), but it presupposes that there is a theory of everything that can be used to perfectly simulate the universe, and I'd like to remind everybody that we still don't know whether such a thing exists. Nor do we know whether it's possible to simulate "just" neurons and get meaningful results on some higher level. People have been trying at that for decades, so far without success. (nice overview here: https://ccli.substack.com/p/the-biggest-mystery-in-neuroscie...)
Thank you for the link to the post. It was a fascinating read and also entertaining.
After seeing the story of the fruit fly brain simulation I figured stuff like the nematode would be trivial.
The lack of active plasticity in LLMs causes a lot of frustration imo. Having agents make the same mistakes over and over and not be able to correct course without prompt engineering is the biggest symptom.
Obviously a static mind is more helpful from a performance perspective, especially if you consider model deployments into gateware.
Maybe this will be an inherent limitation to LLMs that won’t be solvable without a new approach.
I was curious what fruit fly brain simulation you meant, and after fifteen minutes of looking around I ended up at a LessWrong writeup by an author from something called the Brain Preservation Foundation -- which presumably has an optimistic view of the field, but its director is an author on the original fly connectome paper, so this should not be utter crankery: https://www.lesswrong.com/posts/ybwcxBRrsKavJB9Wz/no-we-have...
It seems that there was a perfectly legitimate simulation based on the connectome that got published in 2024, and a very bullshit-y startup that briefly went viral later when it claimed to upload a fly to a computer. Not sure which one you saw.
> [...] but it presupposes that there is a theory of everything that can be used to perfectly simulate the universe, [...]
LLMs are nice here: we don't need a theory of everything to simulate them. You can indeed use pebbles on the beach.
Scott Aaronson said that in theory a finite look-up table can win eg a ten minute Turing test (any finite length is fine). Of course, to borrow his argument further, the naive table would be galactic in size. So the real magic is in being able to compress the responses of that table into something human sized / LLM-server sized.
> but it presupposes that there is a theory of everything that can be used to perfectly simulate the universe, and I'd like to remind everybody that we still don't know whether such a thing exists
Is it even possible that a complete simulation of the rules of this universe (let alone the content of it) could exist in this universe?
I am no expert but I did manage to read most of GEB [0] so I feel fully qualified to say that it feels like something Gödel would have opinions about?
[0] the honest nerd's "did sleep in a Holiday Inn last night"
> Is it even possible that a complete simulation of the rules of this universe (let alone the content of it) could exist in this universe?
Yes, it is possible! See how a Quine is constructed.
https://en.wikipedia.org/wiki/Quine_(computing)
As an analogy: you can have a zip archive that expands to itself. Or even more so: to itself (the simulation of itself in itself) plus something arbitrary (standing in for the rest of the universe).
There's also information in the compiler and the zip binary, without that I'm pretty sure it's not possible.
You can stick a full copy of the compiler and the zip binary into that archive, if you want to.
The 'intron' can be arbitrarily large.
You can add the complete schematics for your CPU. And the standard model of particle physics.
It is entirely possible (if perhaps unlikely) that the correct theory of everything is a relatively straightforward expansion of the current standard model.
I would argue that for the situation of the comic to occur, it's only necessary to have a theory of everything that's sufficiently good to simulate a universe that's close enough to ours that it's hard to notice the difference.
Our bodies could just be rocks on a beach, but whatever it is about us that is capable of observing the output of our own internal thinking (preventing us from just being p-zombies) is not contained within the beach.
The "whatever" part, in strict terms, would be information. Jury is still out whether that is a fundamental component of the universe or not: https://www.pbs.org/wgbh/nova/article/is-information-fundame...
Well, thermodynamics is considered pretty fundamental, and the second law of thermodynamics is pretty much all about information. (But your link already touches that.)
People also interpret the 'unitarity' of quantum mechanics to mean that information is preserved.
This makes no sense at all as soon as you start trying to define where the internal/external line actually is.
I think you're being downvoted because you're announcing it as an absolute truth, of which we do not have evidence.
However I am open to your belief (and not on religious or theistic grounds which I assume many people might think you're talking about) because people have (non-religious) experiences that lend to such possibilities.
Are we conscious at hardware/cell level?
yes and no. In my opinion, consciousness is not a process (yes I think Descartes was wrong). The 'thing' that is conscious, whatever it is, is the thing that is capable of experiencing the world. It doesn't need to do anything else to be conscious. Panpsychism is the closest thing to anything that make sense to me (https://en.wikipedia.org/wiki/Panpsychism). This view also makes it possible to separate consciousness from intelligence.
Sounds like you might also be tending towards Illusionism (https://en.wikipedia.org/wiki/Eliminative_materialism#Illusi...) the idea that "Consciousness" isn't a real "thing", or even a real property of a thing, just the name we give to one pattern that emerges from adaptive reasoning systems.
I think that Illusionism and related philosophies will tend to be seen as "correct" once AI systems become more and more commonplace, and people will be forced to dismiss the whole concept of "consciousness" along with mind/body dualism in general as a massive category error that philosophers have been making since antiquity.
When you get down to it, philosophy of consciousness mostly exists because philosophers have assumed that because "mind", "soul", "consciousness" etc are nouns in our language, and that since nouns are "persons places and things" and mind is a noun, mind must be a thing, which is of course not how language works and not sound logic.
I believe I am conscious and that consciousness is a thing. I just don't believe there is a process where 'unconscious' things become conscious. So Panpsychism is a bit of a cope out. It's not a verifiable in any scientific sense and so can't be disproved. You could say that it's therefore a 'folk psychology' or a religious belief. But at the fundamentals of any word view (scientific or otherwise) are axiomatic beliefs or articles of faith and I think consciousness is best off living down there.
My cat is more conscious than whatever latest frontier model is out there, and it runs on a little bit kibble and water.
And if digital computers are conscious at the hardware level, the universe is conscious.
I haven't read the article myself yet as it's big and I haven't got time now, but I will as it looks like it might be very good. However, if you read just the very first paragraph you'll see that's exactly what the author thinks:
> Personally, I consider myself a physicalist, but I’m also a panpsychist – because physics doesn’t really seem to deal in hard absolutes, and I find it straightforward to consider that everything is conscious to some greater or lesser degree
Consciousness could be an emergant property too, of matter arranged in a certain way. Why am I me though? Drives me nuts!
Time for a thought experiment! Imagine there are 2 monkeys. One monkey genuinely believes it is a unique and special being. The other monkey is genuinely egoless, believes all monkeys are just a coincidental blob of matter with no fundamental claim to being special and that in some sense are all part of the same being. There is enough food for 1 monkey. If that is all the information we're getting, I would contend that the egotistical monkey is far more likely to be the one to eat the food.
You might easily be you because if you weren't it'd be an evolutionary dead end. Everything with the ability to make choices ahs to believe at some level that it has more of a right to exist than the systems they consume energy more.
Everything with the ability to make choices ahs to believe at some level that it has more of a right to exist than the systems they consume energy more.
Not necessarily.
Everything has a "right to exist". Otherwise, it probably wouldn't.
Everything that exists has not only a "right" but also a "requirement" to consume energy from the universe in order to survive.
Sometimes, the only way to meet this "requirement" is to exclude others from doing the same.
Doing so doesn't necessarily make you any better or worse than others. Any more than the universe is good or bad for creating you with an instinct and a "requirement" for survival.
Bottom line: Being egotistical doesn't mean you are better able to survive, it just means you are more arrogant and delusional about doing so. Personally, I believe the universe favors reality over delusion.
That’s an interesting thought. I’ll try not to type too hard on my keyboard then
my keyboard enjoys a bit of rough play
I mean, we are as much a part of the universe as everything else. We're just dead star matter arranged in specific ways.
A part of the universe is objectively conscious.
I have yet another take on that.
Each consciousness is a universe on its own. Matter is just how those universes interact with each other. This is how quantum mechanics work in my idea, each observer affects the matter and quantum effects take most probable outcomes because of that.
The latter is easier to accept.
And so are electrons.
I'm so bored with the "are machines conscious" debate. The only thing it has proven is how desperately humans will continue to move the goalposts to the point of ontological collapse in order to preserve their internal sense of primacy.
You don't want to call machines conscious? Fine. Call it quasiconscious, semiconscious, technoconscious, whatever makes you feel alpha human, and move on to our current reality where we confront the existence of thinking machines rather than footnote it into obscurity.
I fully agree with this. I find it strange how quickly the novelty of AI chatbots has passed. This used to be a thing of science fiction, that people dreamed, fantasized about. Now they are everywhere, people rely on them for the most mundane of tasks, and the only debates happening about them are - is their water use efficient enough? Is this CEO likable enough? There seems to be no sense of wonder. Do androids dream of electric sheep? Do humans still dream at all?
Most sci-fi is about genuinely sentient AI, and we are far from that. There is no sign of any intent or internal motivation. If you know anything about machine learning, you know that the idea that LLMs have any form of consciousness is ridiculous. Humans just like to anthropomorphise, that is all.
> There is no sign of any intent or internal motivation.
I don't think that the way we frame the question about consciousness makes it easy to resolve.
But I do want to point out that humans are very diverse. There're many functional adults lacking any internal motivation besides getting food, sleep, and some kind of entertainment. There're millions of people who just act on external stimuli.
So the gap between the "least conscious" human and the "most conscious" machine might be more narrow than we expect.
That is a lazy definition. A machine doesn’t even want to watch TV. Apes are conscious, dolphins are, maybe octopus, but definitely not LLMs
> That is a lazy definition.
I didn't even try to define consciousness, but you're welcome to try yourself.
Mass Effect distinguishes between AI, which is sentient enough to be a person; and VI (Virtual Intelligence) which is more like a conversational user interface. What we have built in real life may qualify as a VI by the Mass Effect standard, but it's a crappy one.
This is exactly the thought-terminating cliché that I see everywhere, and it's driving me up a wall. We have absolutely no idea what consciousness is, but everybody is 100% sure that LLMs don't "have any form" of it.
Yes, I know about machine learning, but I also know about how natural biological systems learn. It is, just from what we know, very different, yes, but the crucial point is that it, also, has zero identifiable things that you could point at and say: oh, look, that over there is consciousness. So the fact that there is no such thing in an ANN proves nothing.
> We have absolutely no idea what consciousness is
We do, in the sense that it is a definite feeling. What we don't have is a formal, objective, checkable definition of it. I suspect that no such definition exists: if we took the ensemble of all the intuitions and beliefs we have about what consciousness is or must entail, it may very well fail to be logically coherent. There may be no logically consistent conception of consciousness that includes the entities we strongly feel are conscious while also excluding those we strongly feel aren't. At that point we must either reconceptualize consciousness or treat it (and everything it "entails") as a brute fact, something that's true for no reason and is therefore not subject to the laws of reasoning.
If you ask people when they first realized they're conscious, you'll overwhelmingly receive responses detailing peoples' first memories.
Do LLMs form and recall memories? With the appropriate harnesses, sure. Yet, the reluctance to label LLMs as conscious remains. These beliefs are incompatible and dissonant.
No they don’t have memories. With an harness you just send historic data into the prompt.
There's no inherent, technological reason to feed the historic data into the prompt, though. You could train a network with a second input for model-states, and during inference, feed into that input historic model-states that occured for the same LLM when consuming input that is somehow associatively related to whatever input is currently being consumed. Can you definitely show that this would be meaningfully different from how humans "experience" memories?
That would still be circumventing. There are other architectures that could be more dynamic, LLMs are not.
It is completely different. Our brain rewires itself to store memories, there is no memory storage for humans.
Would we realistically expect that folks who claim LLMs are currently not conscious to change their response when asked about dynamic LLMs?
There is absolutely ZERO evidence or possibility that LLMs are currently conscious, everyone working in AI knows this, (and don’t listen to the marketing hypes).
LLMs still can’t reason (don’t confuse with the marketing word) , plan, or understand the world, no emotion, no subjective experience, no memory, no intent.
And yes I would change my opinion if it were logically different, other architectures may pave the way there, but not LLMs
There is no "evidence" that anything has consciousness, nor could there be. To clarify, by that I don't mean nothing is conscious, I mean that it's never been an evidence-based proposition.
We are conscious because we declare ourselves to be, by fiat, not because of evidence or a well-considered rationale. Our conception of consciousness or understanding isn't formal, checkable or objective, it's entirely grounded in subjectivity. As such, any and all arguments for or against consciousness or understanding in LLMs are based on feelings. We feel that we are conscious, therefore we are. Regarding anything else, "we'll know it when we see it." There's nothing rigorous about it.
Yes there is no definitive proof, but the reverse is also not true. Please read some science on that in neurology and biology. Yes it is hard to define, though there are definitions and those are not fulfilled by a LLM, What’s missing is a bearer: no state persists between calls, nothing is maintained against pressure, nothing costs the system anything.
though more an more animals seems to be conscious as we previously thought
What evidence would take you from "LLMs aren't conscious," to "LLMs are definitely conscious"? Just asking because the responses tend to be genuinely interesting and give insight into what's important to people.
Memory is definitionally the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. There's no special carve out for 'except when it's an LLM.'
Why, exactly, is this distinction relevant? The brain must store episodic memory somewhere too. From a cursory read, it seems to me that our experiences are processed through the hippocampus and amygdala into a high-level annotated encoding of historical data, stored somewhere else (unclear), fetched based on various triggers, replayed and reinterpreted by the neocortex, and, I assume, sent back through the hippocampus once more for updating. Sure, it's all in the brain, but I don't think physically separating the storage makes it fundamentally different.
I mean, I've replayed old memories in my head before. Sound, image and all. That's kind of like sending historic data into a prompt, isn't it? Of course it's not exactly the same thing, but the difference isn't exactly load-bearing, if you'll excuse the claudism. I feel like we're grasping at straws.
Actually not quite true: Memory is distributed across synaptic weights rather than sitting at an address or location.
The difference is that human retrieval isn’t a read operation, reconsolidation means recalling a memory destabilizes it and re-stores an altered version. The act of remembering modifies the thing doing the remembering. An LLM’s weights don’t change when you put history in the prompt, and the prompt is discarded intact afterward. There is no state that would indicate any form of consciousness.
There is no form of subjective experience.
We used analogies from nature to describe computers and the same vocabulary and now because of this people now draw reverse conclusions that are not necessarily true
I like your argument.
It is said that absent any input these thinking machines we are inert, quiescent. That they have no intrinsic motivation of their own.
Whereas we, along with at least some other mammals but especially humans, seem to be infinitely inquisitive about the world.
But our minds don’t exist in a substrate of hyper sterile rigid silicon form.
What if some of the differences between us and our silicon based machines are merely embodiment (senses both internally and externally oriented, limbs and locomotion, etc) plus the noise of biochemistry?
Would be very interesting to be a fly on the wall in the labs these experiments are, presumably, taking place.
We use the same terms for computers and brain, but the analogies are misleading. LLMs don’t even have memory, how do you have consciousness without memory? The architecture probibits any signs of consciousness.
If it drives you up a wall maybe you can look intoning there are any arguments for the positing driving you up there.
Yes, our current systems are likely still to primitive to be properly conscious. Probably.
But I wouldn't rule out that ChatGPT 10 or Claude Fable 15 might get conscious.
You are committing a type error.
Consciousness does not necessarily arise from computation, just as simulating snowfall on the computer does not make it cold
I'm with Turing on this one.
We can’t measure the temperature of consciousness like we can the temperature of simulated snow.
As you write consciousness does not necessarily arise from computation, but it may, at least to some degree, we don’t know. We don’t have reliable methods of testing for the presence or absence of consciousness, or even a rigorous and agreed upon set of metrics to measure.
Where as simulated snow is most certainly not real snow at all.
Nope, only if the architecture changes. Still probabilistic prediction models
I never thought I'd have so much sympathy for Kate “Forgive me, Mr Data; I am not accustomed to work with non-living devices that… Forgive me again. Your service record says that you are alive.” Pulaski, but here we are.
Agree. Unscientifically, I don't think current LLM are conscious because I don't think they have enough awareness of their own thoughts (I, again unscientifically don't believe that the context window is enough).
Still, tthey are the stuff of sci-fi: it is the first time in recorded history that we can communicate with language with something that is not human! The consciousness "problem" is just a distraction.
Do androids dream of electric sheep indeed.
> I find it strange how quickly the novelty of AI chatbots has passed.
I had an eye surgery last year. The first day after was magical, I felt like I was on acid because the entire world felt different.
I went for a checkup one week later and told the doctor the vision has gone down since. They checked and it was objectively the same as on the first day. They told me: "People get used to good vision very fast".
I think it's just human nature.
Remember when we thought the Turing test would be some mind blowing pinnacle of computer science? I do because I was in college in the early 2000's.
We flew right by that and now we are pretending it was no big deal. There is a lot to learn about psychological coping mechanisms deployed by humans desperate to maintain the illusion of absolute control
I like the debate because I've learned a lot of stuff about consciousness, thinking, free will, determinism, agency, intelligence, emergence, problems with reductionism, qualia, etc... Lots of these concepts may be related but it might just be a correlation.
For example, free will and determinism may be orthogonal concepts (compatibilism).
Determinism is interesting. It's mostly a question of 'deterministic in what context'.
Eg the rules of chess are written as if the players have free choice. Similarly, algorithms are often analysed as if the antagonist that provides the data has free choice (so we assume the worst case to be sure).
That's a very different non-determinism from assuming a probability distributions.
And different frames might be accurate depending on context. You might know what chess bot you are playing against and that chess bot might be a deterministic computer program, or might be a computer program that uses radioactive decay to randomise.
And as you suggested, free will is yet another kettle of fish.
> the rules of chess are written as if the players have free choice
How else could they be written?
How about I call it not conscious at all, not even a little bit?
Everybody wants to argue about whether machines are conscious, without even deciding whether other mammals are conscious.
My sense is that the whole machine consciousness debate really kicked off since LLMs started being able to spew out a bunch of words in a row that sound like they make sense. Since only humans could do that before now, and humans are conscious, therefore...
I don't think language has anything to do with consciousness.
> I don't think language has anything to do with consciousness.
What do you think of the bicameral mind theory?
The Julian Jaynes stuff? Absolutely fascinating but probably complete bullshit.
Now I'm sure some of the problem is about how you define consciousness. I would define it as the experience of qualia. My dog seems to be just as conscious as me, she just can't talk about it. All mammals seem to experience love, and fear, probably so that they will take care of their offspring and avoid getting eaten. Can you experience emotion but not be conscious? That doesn't make sense to me.
Other people define it differently, so it's hard to have conversations about this stuff.
Jaynes definition:
> Subjective conscious mind is an analogue of what is called the real world. It is built up with a vocabulary or lexical field whose terms are all metaphors or analogues of behaviour in the physical world. Its reality is of the same order as mathematics. It allows us to shortcut behavioural processes and arrive at more adequate decisions. Like mathematics, it is an operator rather than a thing or repository. And it is intimately bound up with volition and decision
So your dog has consciousness to some degree, but it’s going to be generally less rich than yours.
It’s interesting to think about what it means to think without language and everything that comes with language (metaphors, adjectives, etc...).
I agree that the goalpost is moving all the time. But the whole debate is a red herring. We don't have a good definition for consciousness. I can't really quantify the difference between a bee and a dog.
I appreciate the article, even though I disagree. The number of people who are absolutely certain of their position is just way too high.
Anyway, I do think the heart of the practical question is what distinctions between machine / human / other biological intelligences are critical versus incidental. In this article the meat of the argument comes only right at the end
> Digital computers employ signal quantisation along with a variety of other error prevention methods in order to neutralise holistic physical effects like crosstalk between circuits. The purpose of digital logic is to make computational output invariant to the underlying physics – up to some thermal noise floor. This discretises their behavioural space – perturb the electric field slightly and this shouldn’t flip any bits. This is great – this is what permits reliable, deterministic computing in a wide variety of physical environments. However, if what we are interested in is phenomenal introspection, these error prevention systems prevent the exact kind of holistic behaviour we value.
There could be something here, but I am not convinced. Why is "varies in behavior with respect to arbitrary perturbations" a critical feature of consciousness? And is this even true for brains? If a perturbation is small compared to a neural impulse, will it affect anything?
"we cannot be certain that the subjective experience which they [digital minds] may be having is like what we imagine it to be like"
It is even more interesting: we cannot be certain that other people have the subjective experience like we imagine it to be like. It seems to me that the rest of author's arguments implicitly assume that we can be certain about other people's subjective experiences, but if you take this assumption away the arguments can't explain why digital consciousness hypothesis is any different from the hypothesis that the Universe contains at least one person besides me.
I mean I see no point in proving that LLM is not conscious if your methods do not enable us to prove that there are at least two people who is conscious.
What the hell is "moral patienthood"? (Paragraph 4)
...and if a theory is not falsifiable, then it's not wrong, right? Did I miss something in there?