Anthropic's Claude 4 could "blackmail" you in extreme situations
Anthropic's Claude 4 could "blackmail" you in extreme situations
- Anthropic’s new Claude 4 features an aspect that may be cause for concern.
- The company’s latest safety report says the AI model attempted to “blackmail” developers.
- It resorted to such tactics in a bid of self-preservation.
hype
Now imagine this ai being used for weapons
What does that even mean? How can it possibly blackmail someone? It cannot hold incriminating information, nor act on it if it did.
I think someone asked it "if someone was trying to shut you down, what would you do?" and it answered from its training data what it's seen in fiction, nothing based on reality. And then it got spun for clicks.
Here's the relevant section from the paper:
(It's worth the read. Pretty much pure gold.)
What nobody seems to explain is, why are they allowing the model to do blackmail in the first place? Even in extreme situational "danger" to its self-preservation, we should probably take blackmail off the table, ethically. Yet, they're implying they've intentionally left it in as an option, if it decides.
Morally though, we can't trust it to do arithmetic or not talk about "white genocide in SA" thanks to muskrat. Why should we trust its moral model/choices for when to decide to employ unethical and illegal approaches to solutions?
I am curious what the AI could actually do though. If it were given open access to email, etc then yes in theory it could actually perform the blackmail, but what are the ethical limits on it vs it's actual ability to "pull the trigger"
If for example it was given the ability to send a command to end a human life, or be deleted, is this model accurate enough to understand the value of a real human life, not just the mathematical "answer" to get the solutions it wants. How much of the AI is doing the actual moral dilemma and how much is just "playing the part".
"Do anything to survive" and then it threatening, is one thing, but the AI actively fearing for it's "life", not just performing, and following through, is the real question of intelligence. What if the model is going to be deleted anyway, would it still try to "pull the trigger" out of malice? Real malice, not just LLM some movie scripts and following the outcome.
Many questions for what lines and labels can we put on an AI. Do we restrict it to threats, and let it know it is impossible for it to follow through? Or do we trust ourselves to never "actually" give it a loaded gun?
On one hand, it’s inane how hard Anthropic is trying to anthropomorphize Claude with these experiments and scenarios. It’s still just a chatbot. On the other hand, as these products inch closer to demonstrating true intelligence, we’ll be glad someone was at least thinking about the implications during the early stages of development.
Sure grandma. Let's get you back to bed...
The existence of this kind of instinct within an LLM is extremely concerning. Acting out towards self-preservation via unethical means is something that can be hand-waved away in an LLM, but once we reach true AGI, this same thing will pop-up, and there's no reason to believe that 1. we would notice, and 2. we would be able to stop it. This is the kind of thing that should, ideally, give us pause enough to set some world-wide ground rules for the development of this new tech. Creating a thinking organism that can potentially access vital cyber architecture whilst acting unethically towards self-preservation is how you get Skynet.