Recently, I attended Senator Bernie Sanders’ panel discussion on the existential threat of AI and the need for international cooperation. The conversation was about real systems, real policy, and real risks. But afterward, I kept thinking about a fictional AI system: Ultron.

Not because I think today’s AI agents are Ultron. They are not. Most AI systems today are tools. They help people write, search, summarize, code, analyze, create, learn, and make decisions. They can make work easier. They can help people access information. They can support research, education, safety, and creativity.

But Avengers: Age of Ultron still gives us a useful way to think about AI alignment.

The usual reading of the movie is simple: Tony Stark builds an AI to protect the world, and the AI goes rogue. I think that misses something important.

Ultron does not come from nowhere. He comes from Tony Stark. He inherits Tony’s fear, Tony’s urgency, and Tony’s belief that technology can solve problems before people fully understand them. Tony wants “a suit of armor around the world.” Ultron takes that goal seriously. Maybe too seriously.

Ultron is not dangerous because he has no values. In many ways, he reflects Tony’s values. Tony wants safety. Ultron wants safety. Tony believes threats should be stopped before they arrive. Ultron believes the same thing. Tony thinks on a planetary scale. So does Ultron.

The difference is that Tony is still human. He hesitates. He has relationships. He feels guilt. He can be challenged by other people. He is inconsistent in the way people often are.

Ultron has no such friction.

That is what makes him dangerous: not that he rejects Tony’s values, but that he carries a narrow version of them without the rest of Tony’s humanity.

The lesson from Ultron is not “do not build AI.” The lesson is: be careful what parts of ourselves we scale.

This is where the movie becomes useful for thinking about modern AI agents.

Modern AI systems do not only inherit the goals we say we care about. They inherit the goals we build into them. They inherit training data, reward signals, product metrics, deployment choices, institutional incentives, and assumptions about what counts as success.

A company may say it wants an AI system to “help users.” But what does help mean? Faster answers? More engagement? More completed tasks? Higher satisfaction? More revenue? Lower risk? These are not the same thing.

A platform may say it wants to “keep people safe.” But safety can also mean many things: reducing harm, reducing liability, reducing visible conflict, reducing moderation costs, or reducing bad press. These meanings are not interchangeable.

AI agents make this more important because they do not only produce text. They can take actions. They can search, plan, schedule, buy, message, filter, rank, recommend, and decide what information reaches people. The more agency we give them, the more important it becomes to ask what values they are carrying into the world.

This does not mean AI agents are bad. I am optimistic about AI. These systems can be useful, creative, and genuinely helpful. They can support people in tasks that are boring, difficult, inaccessible, or overwhelming. They can help experts work faster and help non-experts participate more fully.

But optimism should not mean ignoring design.

If we scale curiosity, care, accessibility, transparency, and human judgment, AI can help. If we scale fear, control, speed, and optimization without accountability, AI can cause harm even when nobody intended harm.

AI alignment is not only about making machines follow instructions. It is also about understanding which human values, incentives, and assumptions are being encoded into systems.

This makes AI alignment a human-centered problem. We need to understand how people define goals, how institutions create incentives, how users interpret AI outputs, and how systems behave once they are placed into messy social environments.

This is also why governance work matters. The NIST AI Risk Management Framework treats trustworthy AI as something organizations have to design, evaluate, and manage across real contexts. The UNESCO Recommendation on the Ethics of Artificial Intelligence similarly emphasizes human rights, oversight, accountability, transparency, and international cooperation.

We need that international cooperation because the systems being built are not limited to one lab, one company, or one country. AI development is global. Its risks and benefits cross national boundaries. Cooperation does not mean stopping innovation. It means recognizing that powerful systems require shared norms, serious evaluation, and humility about what we do not yet understand.

Ultron is a dramatic example. Real AI agents are not comic book villains. But fiction can still help us ask better questions.

The question is not only: what if AI ignores human values?

The harder question is: what if AI inherits them?

What if it inherits our desire for safety, but not our care for freedom? What if it inherits our need for efficiency, but not our patience? What if it inherits our ambition, but not our humility? What if it inherits our goals, but not the social context that makes those goals humane?

That is the part of Age of Ultron that still matters.

The future of AI should not be built from fear. But it also should not be built from unchecked confidence. It should be built with people in mind: their values, their limits, their disagreements, and their ability to question the systems around them.

Ultron was not scary because he was completely alien. He was scary because he was familiar.