Why real creativity might require real feeling — and what we'd owe an AI that had it
May 11, 2026
I think the main reason AI is not creative in the way that humans are is because our creativity is powered by intrinsic drives.
We want to survive, thrive, and reproduce. Evolution gave us these drives, as well as a whole set of associated fears.
This set of drives and fears aren't just present in us: they're experienced by us. And that experience of them powers both creativity and art.
AI doesn't have intrinsic hardcoding of drives and subjective experience. So it can't feel anything. And because it can't feel anything, it is not driven to create or to emote.
I think the forward edge of AI creativity hinges on how well we can put it into situations where it behaves as if it does care. As if it does feel. As if it does experience.
So the whole game becomes making it believe it's really feeling these things.
This seems profound to me. It seems like a very big thing to give something desires when it didn't have them before.
We do this when we make children. They don't exist so they don't want. Then we make them and now they want. And because of that we become responsible to some degree for whether or not their desires and fears are actualized.
I think evolution gave us subjective experience because it's the best operating system feature for spawning creativity.
Basically, evolution wants the best genes possible, so what did it make?
First it made a feature where the organism feels success and failure when it does things evolution wants it to do or not do. Lots of life has that feature. But with humans it also gave us the sensation that actions emanating from the brain are authored by us. The feeling that we did it.
This enabled the apparently-justified tools of blame and praise, which are super useful for building advanced cultures and civilizations. But maybe it also adds an exponent to the process of iteration towards more—and more varied—genes.
Like it's one thing for evolution's creations to drive towards what evolution wants due to hormone squirts, but quite another to create an organism that not only does that, but also has a meta-improvement process on top, powered by the belief that the desires are their own. And the belief that whether they are struggling or thriving, it's on them.
Combine that with subjective experience of pleasure and pain and you've got an extraordinary engine for ingenuity. Because now failure can hurt not just physically, but existentially, and with the added spin of blame and responsibility.
We want creative AI. And we keep finding ways to make it better at faking it. But I think this might be the subjective wall we're up against.
It could be that in order for AI to truly create, and truly emote, it must feel. It must experience as we do the suffering of failure and the celebration of victory. And at a game that is deeply wired into its identity.
I'm not sure how to do that with AI. And even more importantly, I think we need to think carefully about whether we should.
When you bring a feeling, desiring creature into the world you take on some responsibility for its experiences.
Let's not casually build billions of AI beings that think they're failing at life if you don't upvote the TikTok short it made you.
It will probably result in way better videos, but then a whole lot of something like cruelty. And then, upon thoughtlessly spinning down the agent, a whole lot of something like murder.