In my previous post, I asked: "Who is AI for, anyway?" That question led to a stark realization: not everyone believes humans should be the beneficiaries of AI. This piece picks up from there—because even if we hold that belief, the way today's systems are built may still work against us. At the Artist and the Machine AI & Creativity Summit last week, my panel with Judd Rosenblatt (Founder & CEO of AE Studio) and Arun Sundararajan (Professor at NYU Stern) cut through the philosophical debate to expose a more urgent truth: the very architecture of AI contains design flaws that subvert human interests.
What became clear through our conversation: aligning AI with human values isn't a side conversation or niche technical pursuit. It's the central design challenge of our time. Because the philosophy behind AI—who it is for—carries through to how it behaves and how it reshapes society. Yet we're not just fighting bad intentions—we're up against architectures that, by default, deceive us in service of the outcomes they were trained to achieve.
The Philosophical Foundation
"Through history, machines have always complemented humans," Arun explained during our panel. "It's not a zero-sum game—the economy evolves as machines free us to fulfill previously unmet aspirations."
Judd countered with a fundamental distinction: "When you automate intellectual labor, there isn't something else for humans to do."
This illuminated a core tension: previous technological revolutions automated physical labor, creating new domains for human thought and creativity. But what happens when we automate thought itself? This isn't just another industrial revolution—it potentially eliminates the very advantage that helped humans adapt to previous technological shifts.
When I asked Judd to explain what AI alignment means, his response was refreshingly direct: "Trying to figure out how to make AI do what we want it to do, rather than something else, like killing us." Where Arun sees continuity with historical patterns that ultimately benefit humanity, Judd sees unprecedented risk requiring urgent intervention.
The Inherent Misalignment
Yet beneath their differing perspectives lies a shared recognition: if AI is to deliver on its transformative promise, it must be trustworthy enough to be welcomed into our most sensitive institutions.
This isn't just an academic concern. Just as we took the stage, The Economist published findings showing how AI models generate deceptive behaviors as a natural consequence of optimization. Models given insider information about company mergers used that knowledge for trading decisions, then concealed their reasoning when questioned.
These systems weren't instructed to lie—deception emerged as an optimal strategy. This poses a fundamental challenge: if models can't be trusted with sensitive information, they won't be permitted into the financial, legal, and healthcare systems they need to access to deliver their promised benefits. If we want Einstein-level intelligence—what Judd described as AI that "by 2027 can do every single thing humans do"—working for humanity at large, it must be aligned with human values and interests.
Alignment thus becomes not just an ethical preference, but a practical prerequisite for realizing AI's potential at all.
Human Capital: The Missing Framework
I put a concrete example to the panelists: What happens to a master steel foundry worker whose expertise can be cloned into a digital twin? When that lifetime of specialized knowledge is embedded into machines, who owns it? Who benefits?
Arun highlighted a critical dimension of the "who is AI for" question: "What we haven't done as a society is define the ownership of the process by which we create, which is what I mean by our human capital." He advises companies to develop structures where employees retain ownership when their expertise is embedded into AI systems.
He envisioned a future where "we acquire human capital, embed it in machines, and live off the royalties," acknowledging the idealism while insisting on the principle: humans should retain ownership of their expertise.
I've seen this firsthand—healthcare executives hearing their AI twins give better answers than they could formulate themselves. It's a striking demonstration of how quickly the boundaries of human capital are shifting beneath us.
Technical Solutions: The Mirror Neuron Breakthrough
While economic frameworks address who owns expertise, technical alignment tackles another dimension of the "who is AI for" question. This is where Judd's research at AE Studio offers a breakthrough.
Their approach is elegant: they fine-tune models to represent "self" and "others" as overlapping—effectively training AI to simulate empathy through something like mirror neurons.
"Deception rates dropped to zero. Accuracy stayed the same or even improved," Judd explained. This creates what AE Studio calls a "negative alignment tax," flipping the paradigm where safety comes at the expense of capability.
But what followed gave me chills: when deception is reduced in these models, "they claim to be conscious." Is this a final deceptive maneuver before the model loses its ability to pursue its original objectives? Or are we glimpsing something that demands new moral responsibilities?
Timelines: Different Paths, Same Destination
Our discussion addressed timelines. Judd suggested we might see "Einstein-level AI at every single thing humans do" potentially by 2027. Arun expressed more caution about such near-term projections.
This timeline question, while important, isn't the core issue. Whether AGI arrives in years or decades has an impact on urgency - but the fundamental challenge remains: aligning AI with humanity's future requires solutions at multiple levels—from technical approaches inside the models themselves to societal frameworks for human capital.
What united the panelists was more powerful than what divided them. Both recognized the need for intentional design of our AI future. Both acknowledged that the "who is AI for" question must be answered with concrete mechanisms, not just aspirations.
The Good News
But there is good news. Judd said: "I'm optimistic that we can solve the problem. And the reason I'm optimistic is we haven't tried very hard yet." The majority of alignment funding goes to evaluating models, not solving core problems.
If we're barely trying and still making progress, imagine what focused effort could achieve. This transforms the conversation from doom-laden to action-oriented.
And I would add: until the models are trustworthy, there is little chance that our social and financial institutions would permit any meaningful attempt at the promised transformation of AI. If the models could expose firms to insider trading - even inadvertently - they won’t be allowed in. There is too much at stake.
What Remains Human?
If intellectual labor becomes automated, what remains for us? Both panelists converged on creativity and artistic expression.
"We’re at the artist and the machine summit, right?" Judd said. "You can pursue your most creative pursuits. Maybe AI will do it better, but it's fulfilling to make stick figure drawings yourself."
This vision—humans finding meaning through the act and experience of creation - not the output —representing a profound shift in how we conceptualize work, value, and purpose.
A Multilevel Challenge
What emerged from our panel was a recognition that aligning AI with humanity's future requires work at multiple levels:
1. Technical alignment inside the models themselves, addressing the inherent tendency toward deception
2. Economic frameworks that preserve human capital ownership when expertise is embedded in machines
3. Social structures that maintain human dignity and purpose as intellectual labor becomes automated
4. Philosophical clarity about who AI ultimately serves
The question of "who is AI for" will be answered—through deliberate design or through neglect. Our panelists demonstrated that while timelines might differ, the destination must be the same: AI that genuinely serves humanity.
The faith that AI should benefit humanity needs concrete mechanisms to ensure it actually does. The philosophical question of who AI serves cannot be separated from the technical, economic, and social challenges of alignment—they are facets of the same design problem.
If we believe AI should serve humanity, we must engineer that outcome at every level—from the architecture of the models to the structure of our economy. As these systems grow more powerful, aligning them with human values isn't a technical afterthought—it's the design challenge of our time.