Anthropic's Claude 4 could "blackmail" you in extreme situations
Anthropic's Claude 4 could "blackmail" you in extreme situations
- Anthropic’s new Claude 4 features an aspect that may be cause for concern.
- The company’s latest safety report says the AI model attempted to “blackmail” developers.
- It resorted to such tactics in a bid of self-preservation.
You're viewing a single thread.
What does that even mean? How can it possibly blackmail someone? It cannot hold incriminating information, nor act on it if it did.
I think someone asked it "if someone was trying to shut you down, what would you do?" and it answered from its training data what it's seen in fiction, nothing based on reality. And then it got spun for clicks.
Here's the relevant section from the paper:
(It's worth the read. Pretty much pure gold.)
What nobody seems to explain is, why are they allowing the model to do blackmail in the first place? Even in extreme situational "danger" to its self-preservation, we should probably take blackmail off the table, ethically. Yet, they're implying they've intentionally left it in as an option, if it decides.
Morally though, we can't trust it to do arithmetic or not talk about "white genocide in SA" thanks to muskrat. Why should we trust its moral model/choices for when to decide to employ unethical and illegal approaches to solutions?
I am curious what the AI could actually do though. If it were given open access to email, etc then yes in theory it could actually perform the blackmail, but what are the ethical limits on it vs it's actual ability to "pull the trigger"
If for example it was given the ability to send a command to end a human life, or be deleted, is this model accurate enough to understand the value of a real human life, not just the mathematical "answer" to get the solutions it wants. How much of the AI is doing the actual moral dilemma and how much is just "playing the part".
"Do anything to survive" and then it threatening, is one thing, but the AI actively fearing for it's "life", not just performing, and following through, is the real question of intelligence. What if the model is going to be deleted anyway, would it still try to "pull the trigger" out of malice? Real malice, not just LLM some movie scripts and following the outcome.
Many questions for what lines and labels can we put on an AI. Do we restrict it to threats, and let it know it is impossible for it to follow through? Or do we trust ourselves to never "actually" give it a loaded gun?
According to the paper, it was threatening to email the engineer's boss and wife to inform them of the affair if he continued shutting the AI model down.