Digital tools, including artificial intelligence (AI), are increasingly becoming mechanisms of sexual violence and of other forms of misogyny. This digital sexual violence threatens women in new ways because the tools that enable it are becoming more accessible and because of the public and permanent nature of the abusive material once it has been circulated online.

This is one aspect of a broader problem — that technological development does not necessarily parallel social progress and in fact often reproduces traditional models of subjugation. Implementing technology in liberatory ways takes conscious and deliberate effort on the part of technology developers and regulators.

The rise of digital sexual violence and abuse of AI

The majority of undergraduate students in Canada have reported being subject to digital sexual violence, including harassment, image-based sexual abuse, sexual aggression and coercion, and sex- or sexuality-based hate speech.

Digital sexual violence impacts women and girls at significantly higher rates than it does men and boys, and perpetrators in both cases are overwhelmingly men and boys. Young women, racialized women, and Indigenous women are further more likely to be victimized.

Though we often discuss the virtual world as if it is completely separate from “real” life, that is a misunderstanding of the relationship between online and offline experiences. New modes of harming people are easily added to abusive offline relationships by means of novel technology and online tools. And online and technologically-facilitated misogyny, like violent pornography, can bleed into the physical world and lead to increased physical violence, including rape, domestic and intimate partner violence, and femicide, according to Hannah Tonkin, the Women’s Safety Commissioner in the Australian state of New South Wales. As well, Ontario’s Centre for International Governance Innovation reports that digital sexual violence can lead to physical sexual violence, and some victims alter their offline behaviour due to fear of impending violence resulting from online abuse. In all these ways and more, virtual and offline misogyny can feed into and reinforce each other.

The latest wave of AI makes all of this worse. Though the word “intelligence” remains, for the moment, an exaggeration of their capabilities, algorithms are trained by being fed a high volume of existing content through which they become able to rapidly and easily complete certain kinds of complex tasks. Because the existing content is created, primarily, by humans with pre-existing biases, AI tools will often mirror and reproduce the same oppressions that permeate our society — such as misogyny and racism.

AI tools make creating digitally-altered sexual images much faster and easier. In one case, a 14-year-old girl in the UK died by suicide after male students used AI to doctor images of her face onto the bodies of pornography performers and circulated them on social media. In another case, students from a Winnipeg high school approached school officials regarding several personal photos explicitly altered with AI.

Technology developers and social media companies often lobby against regulation, and enthusiasm about corporate innovation tends to override discussions regarding the harmful social impacts of loosely-regulated tech development. This laissez-faire attitude backed corporate power prevents us from critically interrogating how AI tools work, how they often get used, and the role that they play in reproducing injustice. So most of the time, when problems of digital sexual violence or abuse of AI arise, they are addressed as individual rather than socially organized, collective concerns.

Digitally narratives of physical sexual violence as a model

Many victims of sexual violence have responded by deploying their own digitalized narratives — narratives rooted in feminist analysis which publicize personal experiences with sexual violence by using the language and style of the digital platform in question.

In 2014 on Twitter, now X, two Canadian women journalists started the hashtag #BeenRapedNeverReported in response to widespread public accusations that victims of notorious radio host Jian Ghomeshi’s sexual assault were lying. The hashtag gained popularity among many women — and some men — as a tool to narrate their own stories regarding their hesitation to report sexual violence to authorities.

Ten years later, we can look to this democratized, digital form of narrativizing physical sexual violence as a model for narrativizing digital sexual violence. The historical feminist method demands publicizing accounts of sexual violence to both challenge it and conceptualize alternative methods of justice — while connectivity and virality are harmful for spreading digital violence, they can be reappropriated to challenge traditional misogynist narratives around sexual violence.

Cultural measures to combat digital sexual violence

Apart from expanding existing laws and regulations to fully account for the novel aspects of specifically digital sexual violence, there are measures we can take to alleviate both its prevalence and harm.

Schools across Canada largely fail to address the full context of digital sexual violence. While technofeminist academia regularly discusses such subjects, high school curricula lag behind, compromising young people’s digital safety in an increasingly virtual world. Education about digital sexual violence is often lumped in with traditional cyberbullying, neglecting to contextualize the sex- and sexuality-based nature of the violence, the power imbalances, what exactly constitutes digital sexual violence, and the legal consequences of violating someone’s agency. Those most at risk of experiencing digital sexual violence are thus generally unaware of the context within which it occurs, and of their rights. As such, a critical part of moving forward with safety and freedom in mind is adapting high school curricula to reflect detailed and contextualized accounts of digital and technology-facilitated sexual violence, thereby enabling young people to widen their knowledge and understand their agency.

It is also critical to address the violent sexual content young people are exposed to online. Such content serves as sexual scripts, encouraging young people to reenact in real life the dynamics they absorb from the virtual world. There continue to be debates about how best to intervene to reduce the harms caused by online content, but many feminists advocate for restricting violent pornography by age. And while it is assumed that older people are less prone to being impacted by the content they consume than younger people, we must move beyond the simplistic, binary debate that results from understanding such issues solely in terms of freedom of expression and into a more grounded discussion about the relationship between violent pornography and material harms.

Technological measures to combat sexual violence

It is, again, important to be clear that when it comes to AI's role in this, the problem is not merely a feature of how people use AI tools, but rather is a product of how those tools work – it is built into the data used to train them, the process of that training, and other features of their design. Most of the time, with current implementations of AI, all of this results in skewed algorithms and search results, and other outputs that reproduce existing social biases.

It does not have to be this way, however. We can actively make improvements in the design of digital tools. Rather than implementing solutions like content moderation and reporting assistance as an afterthought, we can plan technology with the existing status quo in mind, proactively designing it to offer features that seek to educate and assist potential victims. We can also train AI to recognize abusive material and encourage it to erase the material or block the user spreading it.

Some scholars suggest training AI less stringently, so that it is able to go beyond predetermined or dominant — and therefore often prejudiced — patterns of thought. Allowing AI to train itself with less human interference can offer novel ways of interacting with digital tools. It might be possible for AI to catch patterns that human researchers may miss. For example, within the framework of social media, it can observe comments left on public forums and use the information to detect users likely to create or distribute violating material.

The dialogue surrounding the equitable development of technology is complex, and navigating concerns related to censorship, freedom of expression, and making use of technology to its full capacity is not easy. But it’s important to move forward with an approach to technological development that prioritizes their impacts on people — not the profit of tech conglomerates. When corporations actively reject regulation that seeks to protect vulnerable populations from the negative effects of their technologies, we must take measures to ensure that those populations’ needs are centred in future development.

What can grassroots movements do?

Addressing digital sexual violence and sexual violence more broadly also means supporting a broad range of grassroots feminist interventions. Movements can, for instance, engage in popular education campaigns that draw on literature and real-world experience to build digital literacy and craft a culture of communication and trust so affected people are encouraged to come forward. Being accessible and safe for impacted people is a necessary first step to any meaningful work to create change.

Movements can also collect data from the public, analyze and interpret it using appropriate intersectional feminist frameworks, and campaign for change based on that grassroots research. Femicide organizations across the world — like the Femicide Watch Platform — have used similar methods to encourage people to recognize certain dynamics and crimes as uniquely motivated by prejudiced attitudes.

In addition, feminist organizations can continue the work of supporting survivors, which has always been central to feminist responses to the problem — offering material support, providing legal assistance to victims, making referrals to other resources, and so on.

By combining technological measures informed by feminist analysis with grassroots feminist interventions — specifically those that seek to venture beyond traditionally carceral approaches — we can ensure that our efforts to address gendered violence will be able to meet the ever-evolving face of the problem.

Please subscribe to The Media Co-op's newsletter!