Kids are making deepfakes of each other, and laws aren’t keeping up
Kids are making deepfakes of each other, and laws aren’t keeping up
Schools and lawmakers are grappling with how to address a new form of peer-on-peer image-based sexual abuse that disproportionately targets girls.
You're viewing a single thread.
Honestly I think we need to understand that this is no different to sticking a photo of someone's head on a porn magazine photo. It's not real. It's just less janky.
I would categorise it as sexual harassment, not abuse. Still serious, but a different level
Schools generally means it involves underage individuals, which makes any content using them csam. So in effect, the "AI" companies are generating a ton of csam and nobody is doing anything about it.
Do deepfake explicit images created from a non-explicit image actually qualify as CSAM?
I would consider that as qualifying. Because it's targeted harassment in a sexually-explicit manner. All the girl would have to do is claim it's her.
Source: I'm a father of teenage daughters. I would pursue the individual(s) who started it and make them regret their choices.
I don't know personally. The admins of the fediverse likely do, considering it's something they've had to deal with from the start. So, they can likely answer much better than I might be able to.
Drawing a sexy cartoon that looks like an adult, with a caption that says "I'm 12", counts. So yeah, probably.
Disagree. Not CSAM when no abuse has taken place.
That's my point.
Except, you know, the harassment and abuse of said deepfaked individual. Which is sexual in nature. Sexual harassment and abuse of a child using materials generated based on the child's identity.
Maybe we could have a name for it. Something like Child-based sexual harassment and abuse material... CSHAM, or maybe just CSAM, you know, to remember it more easily.
I think generating and sharing sexually explicit images of a person without their consent is abuse.
That's distinct from generating an image that looks like CSAM without the involvement of any real child. While I find that disturbing, I'm morally uncomfortable criminalizing an act that has no victim.
Harassment sure, but not abuse.
If someone put a camera in the girls’ locker room and distributed photos from that, would you consider it CSAM? No contact would have taken place so the kids would be unaware when they were photographed, is it still abuse?
If so, how is the psychological effect of a convincing deepfake any different?
If someone puts a camera in a locker room, that means that someone entered a space where you would usually feel safe. It implies the potential of a physical threat.
It also means that someone observed you when you were doing "secret" things. One may feel vulnerable in such situations. Even a seasoned nude model might be embarrassed to be seen while changing, maybe in a dishevelled state.
I would think it is very different. Unless you're only thinking about the psychological effect on the viewer.
There's a thing that was happening in the past. Not sure it's still happening, due to lack of news about it. It was something called "glamour modeling" I think or an extension of it.
Basically, official/legal photography studios took pictures of child models in swimsuits and revealing clothing, at times in suggestive positions and sold them to interested parties.
Nothing untoward directly happened to the children. They weren't physically abused. They were treated as regular fashion models. And yet, it's still csam. Why? Because of the intention behind making those pictures.
The intention to exploit.
Yes, finding out that your peers have been sharing deep fake pornography of you is absolutely fine and a normal thing for young girls to go through in school. No girls have ever killed themselves because of this exact sort of thing, surely. This definitely will not add in any way to the way women and girls are made to feel entirely disgustingly dehumanized by every man or boy in their lives. Groups of men and boys reducing them and their bodies down to vivid sexual fantasies that they can quickly generate photo realistic images of.
If the person in the image is underaged then it should be classified as child pornography. If the woman who's photo is being used hasnt consented to this then it should be classified as sexual exploitation.
Women and girls have faced degrees of this kind of sexual exploitation by men and boys since the latter half of the 20th century. But this is a severe escalation in that behavior. It should be illegal to do this and it should be prosecuted when and where it is found to occur.
It's bullying with a sexual element. The fact that it uses AI or deepfakes is secondary, just as it was secondary when it was photoshop, just as it was secondary when it was cutting out photos. It's always about using it bully someone.
This is different because it's easier. It's not really different because it (can be) more realistic, because it was never about being realistic, otherwise blatantly unrealistic images wouldn't have been used to do it. Indeed, the fact that it can be realistic will help blunt the impact of the leaking of real nudes.
It's sexually objectifying the bodies of girls and turning them into shared sexual fantasies their male peers are engaging in. It is ABSOLUTELY different because it is more realistic. We are talking about entire deep fake porngraphy production and distribution groups IN THEIR OWN SCHOOLS. The amount of teenage boys cutting pictures out and photoshopping them was nowhere near as common as this is fast becoming and it was NOT the same as seeing a naked body algorithmically derived to appear as realistic as possible.
Can you stop trying to find a silver lining in the sexual exploitation of teenage girls? You clearly don't understand the kinds of long term psychological harm that is caused by being exploited in this way. It was also exploitative and also fucked up when it was in photoshop, this many orders of magnitude more sophisticated and accessible.
Youre also wrong that this is about bullying. Its an introduction to girls being tools for male sexual gratification. It's LITERALLY commodifiying teenage girls as sexual experiences and then sharing them in groups together. It's criminal. The consent of the individual has been entirely erased. Dehumanization in its most direct form. It should be against the law and it should be prosecuted very seriously wherever it is found to occur.
If a boy fantasises sexually about a girl, is that harmful to her? If he tells his friends about it? No, this is not harmful - these actions do not affect her in any way. What affects the girl is how the boys might then treat her differently than they would do someone they don't find sexually attractive.
The solution, in both cases, has to be to address the harmful behaviour. The only arguments for criminalising deepfakes themselves are also arguments for criminalising sexual fantasies. that is why people are talking about thought crime, because once you criminalise things that are harmless on their own, but which might down the line lead to directly harmful behaviour, there is no other distinction.
The consent of the individual has been entirely erased. Dehumanization in its most direct form.
Both of these, for example, apply just as readily to discussing a shared sexual fantasy about someone who didn't agree to it.
No distinction, that is, other than this is new and icky. I don't want government policy to be dictated by fear of the new and by what people find icky, though. I do lots of stuff people find icky.
No an image that is shared and distributed is not the same as a fantasy in someone's head. That is deranged. Should CSAM also be legal because making it illegal is like criminalizing the fantasies of pedophiles? Absolutely insane logical framework you have there.
This isnt fantasy. It is content. It is media. It is material. It is produced without the consent of the girls and women being sexualized and it commodifies their existence, literally transforming the idea of them into sexual media consumed for the gratification of boys and men.
It is genuinely incredible to me that you could be so unempathetic, so impassive, so detached from the real world and the consequences of this, that you could even make this comparison. You have seemingly no idea what youre talking about if you believe that pornography is the same thing as mental fantasies.
And even in the case of mental fantasies, are those all good? Is it really a good thing that boys see the mere existence of the girls around them as inherently some kind of sexual availability?
When someone makes child porn they put a child in a sexual situation - which is something that we have amassed a pile of evidence is extremely harmful to the child.
For all you have said - "without the consent" - "being sexualised" - "commodifies their existence" - you haven't told us what the harm is. If you think those things are in and of themselves harmful then I need to know more about what you mean because:
- if someone thinks of me sexually without my consent I am not harmed
- if someone sexualises me in their mind I am not harmed
- I don't know what the "commodification of one's existence" can actually mean - I can't buy or sell "the existence of women" (does buying something's existence mean the same as buying the thing, or something else?) the same I can aluminium, and I don't see how being able to (easily) make (realistic) nude images of someone changes this in any way
It is genuinely incredible to me that you could be so unempathetic,
I am not unempathetic, but I attribute the blame for what makes me feel bad about the situation is that girls are being made to feel bad and ashamed not that a particular technology is now being used in one step of that.
I am just genuinely speechless than you seemingly do not understand how sickening and invasive it is for your peers to create and share sexual content of you without your consent. Yes its extremely harmful. Its not a matter of feeling ashamed, its a matter of literally feeling like your value to the world is dictated by your role in the sexualities of heterosexual boys and men. It is feeling like your own body doesnt belong to you but can be freely claimed by others. It is losing trust in all your male friends and peers, because it feels like without you knowing they've already decided that you're a sexual experience for them.
We do know the harm of this kind of sexualization. Women and girls have been talking about it for generations. This isnt new, just a new streamlined way to spread it. It should be illegal. It should be against the law to turn someone's images into AI generated pornography. It should also be illegal to share those images with others.
Its not a matter of feeling ashamed, its a matter of literally feeling like your value to the world is dictated by your role in the sexualities of heterosexual boys and men. It is feeling like your own body doesnt belong to you but can be freely claimed by others. It is losing trust in all your male friends and peers, because it feels like without you knowing they’ve already decided that you’re a sexual experience for them.
Why is it these things? Why does someone doing something with something which is not your body make it feel like your body doesn't belong to you? Why does it not instead make it feel like images of your body don't belong to you? Several of these things could equally be used to describe the situation when someone is fantasised about without their knowledge - why is that different? In Germany there's a legal concept called "right to one's own image" but there isn't in many other countries, and besides, what you're describing goes beyond this.
My thinking behind these questions is that I cannot see anything inherent, anything necessary about the creation of fake sexual images of someone which leads to these harms, and that instead there is an aspect of our society which very explicitly punishes and shames people - woman far more so than men - for being in this situation, and that without that, we would be having a very different conversation.
Starting from the position that the harm is in the creation of the images is like starting from the position that the harm of rape is in "defiling" the person raped. Rape isn't wrong because it makes you worthless to society - society is wrong for devaluing rape victims. Society is wrong for devaluing and shaming those who have fake images made of them.
We do know the harm of this kind of sexualization. Women and girls have been talking about it for generations. This isnt new, just a new streamlined way to spread it. It should be illegal.
Can you be more explicit about what it's the same as?
The sexualization of women and girls is pervasive across literally every level of western culture. What do you think the purpose is of the victims head and face being in the image? Do you believe that it plays an incidental and unrelated role? Do you believe that finding out that, there is an entire group of people who you thought were your friends but are in actuality taking pictures of your head and masturbating to the idea of you performing sex acts for them using alorthimically derived likenesses of your naked body, has no psychological consequences for you whatsoever? I'm just talking about it and it makes me want to throw up. It is a fucking nightmare. This is not normal. This is not creating a healthy relationship with sexuality and it is enforcing a view of women and their bodies existing for the gratification of men.
You continuously attempt to extrapolate some very bizarre metaphors about this that are not at all applicable. This scenario is horrifying. Teenage girls should not be subject to scenarios like this. It is sexual exploitation. It is dehumanization. It promotes misogynistic views of women. This is NOT a matter of sexual liberation. Youre essentially saying that men and boys can't be expected to treat girls and women as actual people and instead must be allowed to turn their friends and peers into fetishized media content they can share amongst each other. Thats fucking disgusting. The longer you talk the more you start to sound like an incel. I'm not saying you are one, but this is the kind of behavior that they defend.
Do you believe that finding out that, there is an entire group of people who you thought were your friends but are in actuality taking pictures of your head and masturbating to the idea of you performing sex acts for them using alorthimically derived likenesses of your naked body, has no psychological consequences for you whatsoever?
Do you think the consequences of finding out are significantly different than finding out they're doing it in their imagination? If so, why?
Youre essentially saying that men and boys can’t be expected to treat girls and women as actual people and instead must be allowed to turn their friends and peers into fetishized media content they can share amongst each other.
And, just to be clear, by this you mean the stuff with pictures, not talking or thinking about them? Because, again, the words "media content" just don't seem to be key to any harm being done.
Your approach is consistently to say that "this is harmful, this is disgusting", but not to say why. Likewise you say that the "metaphors are not at all applicable" but you don't say at all what the important difference is between "people who you thought were your friends but are in actuality taking pictures of your head and masturbating to the idea of you performing sex acts for them using alorthimically derived likenesses of your naked body" and "people who you thought were your friends but are in actuality imagining your head and masturbating to the idea of you performing sex acts for them using imagined likenesses of your naked body". Both acts are sexualisation, both are done without consent, both could cause poor treatment by the people doing it.
I see two possiblities - either you see this as so obviously and fundamentally wrong you don't have a way of describing way, or you know that the two scenarios are fundamentally similar but know that the idea of thought-crime is unsustainable.
Finally it's necessary to address the gendered way you're talking about this. While obviously there is a huge discrepancy in male perpetrators and female victims of sexual abuse and crimes, it makes it sound like you think this is only a problem because, or when, it affects women and girls. You should probably think about that, because for years we've been making deserved progress at making things gender-neutral and I doubt you'd accept this kind of thing in other areas.
Are you OK with sexually explicit photos of children taken without their knowledge? They’re not being actively put in a sexual situation if you’re snapping photos with a hidden camera in a locker room, for example. You ok with that?
The harm is:
- Those photos now exist in the world and can lead to direct harm to the victim by their exposure
- it normalizes pedophilia and creates a culture of trading images, leading to more abuse to meet demand for more images
- The people sharing those photos learn to treat people like objects for their sexual gratification, ignoring their consent and agency. They are more likely to mistreat people they have learned to objectify.
- your body should not be used for the profit or gratification of others without your consent. In my mind this includes taking or using your picture without your consent.
Are you OK with sexually explicit photos of children taken without their knowledge? They’re not being actively put in a sexual situation if you’re snapping photos with a hidden camera in a locker room, for example. You ok with that?
No, but the harm certainly is not the same as CSAM and it should not be treated the same.
- it normalizes pedophilia and creates a culture of trading images, leading to more abuse to meet demand for more images
- The people sharing those photos learn to treat people like objects for their sexual gratification, ignoring their consent and agency. They are more likely to mistreat people they have learned to objectify.
as far as I know there is no good evidence that this is the case and is a big controversy in the topic of fake child porn, i.e. whether it leads to more child abuse (encouraging paedophiles) or less (gives them a safe outlet) or no change.
your body should not be used for the profit or gratification of others without your consent. In my mind this includes taking or using your picture without your consent.
If someone fantasises about me without my consent I do not give a shit, and I don't think there's any justification for it. I would give a shit if it affected me somehow (this is your first bullet point, but for a different situation, to be clear) but that's different.
Hm. I wasn’t expecting the pro-child porn argument. All I can say is that’s absolutely legally and morally CSAM, and you’re fuckin nasty. Oof. Not really gonna bother with the rest because, well, yikes.
Hey, it's OK to say you just don't have any counter-argument instead of making blatantly false characterisations.
Historically, the respectability of a woman depended on her sexuality. In many conservative cultures and communities, that is still true. Spreading the message that deepfakes are some particular horrible form of harassment reinforces that view.
If having your head on the model of a nude model is a terrible crime, then what does that say about the nude model? What does it say about women who simply happen to develop a larger bosom or lips? What does it say about sex before marriage?
The implicit message here is simply harmful to girls and women.
That doesn't mean that we should tolerate harassment. But it needs to be understood that we can do no more to stop this kind of harassment than we can do to stop any other kind.
This is just apologia for the sexual commodification and exploitation of girls and women. There literally is no girl being sexually liberated here, she has literally had the choice taken from her. Sexual liberation does NOT mean "boys and men can turn all women into personal maturation aids". This ENFORCES patriarchy and subjugation of women. It literally teaches girls that their bodies do not belong to them, that its totally understandable for boys to strip them of humanity itself and turn them into sex dolls.
The most deepfaked women are certainly actresses or musicians; attractive people that appear on screens and are known by much of the population.
In some countries, they do not allow people to appear on-screen exactly because of that. Or at least, that's one justification. If the honor or humanity of a woman depends on sexual feelings that she might or might not arouse in men, then women cannot be free. And men probably can't be free either.
At no point have I claimed that anyone is being liberated here. I do not know what will happen. I'm just pointing out how your message is harmful.
Spoken like someone who hasn't been around women.
You mean like a nerd who reads too much?
Thank you. Focusing on the harm the victims is the right way to understand this issue. Too many people in here hunting for a semantic loophole.
I'm not even going to begin describing all the ways that what you just said is fucked up.
I'll just point out that online deepfake technology is FAR more accessible to the average 13 year old to use on their peers than "porno mags" were in our day.
You want to compare taking your 13 year old classmates photo off of Facebook, running it through an AI and in five seconds creating photo-realistic adult content featuring them, and compare that to getting your dad's skin-mag from under his mattress when he's not home, cutting your classmates face out of a yearbook, taping it on, then sneaking THAT into the computer lab at school so that you can photocopy it and pass it around in home room, and then putting the skin-mag BACK under the mattress before your dad finds out.
Is that right...is THAT what you're trying to say? Are those the two things that you're trying say are equivalent?
Yes, we all know it's fucked up. The point is that we don't need a new class of laws just because it's harassment and bullying ✨with AI✨.
I hope it might lead to a situation of dirty pics/vids not being a problem for the people in it any more, as it could be a deepfake. Like there were cases where a surfacing dirty pic was used for blackmail, ruined someones career or got them kicked out of some comittee, but since it could be fabrication now, I hope this will beva thing of the past, soon.
That could be a socially healthy place to end up at. I don't see it anytime soon though. Just look at the other response I got.
Sure. That might end up being a socially healthy place for adults to end up.
But it will never work that way for young teens. Their brains aren't done baking yet. They don't have the emotional maturity to understand that enough to be "okay with it because it's just a fake".
That's why we protect kids rather than just telling them "hey it's okay...it's only a fake."
Furthermore, we generally assume malicious intent, but I wouldn't be surprised if teenagers were using the app to 'get' big boobs etc., we all have seen those shopped pictures with deformed background 😁