AI-Generated Disinformation
What is AI-Generated Disinformation?
AI-Generated DisinformationFalse or misleading content produced or amplified by generative AI to deceive audiences, manipulate opinion, or influence elections, markets, or conflicts.
AI-generated disinformation uses large language models, image generators, and voice or video synthesis to mass-produce convincing fake news, social-media posts, fabricated interviews, and synthetic evidence. Compared with traditional propaganda, generative AI lowers the cost of fluent multilingual content, enables on-demand personalisation per target audience, and allows fast iteration after fact-checks appear. Documented cases include AI-generated robocalls impersonating political candidates during the 2024 elections and synthetic videos used in influence operations around the war in Ukraine. Mitigations span content-provenance standards (C2PA), platform-level detection, watermarking of synthetic media, media-literacy education, and legal regimes such as the EU AI Act's transparency rules for deepfakes.
● Examples
- 01
AI-cloned robocalls impersonating a presidential candidate to dissuade voters from going to the polls in the 2024 US primaries.
- 02
Coordinated networks of LLM-written social-media accounts amplifying a fabricated leak during a geopolitical crisis.
● Frequently asked questions
What is AI-Generated Disinformation?
False or misleading content produced or amplified by generative AI to deceive audiences, manipulate opinion, or influence elections, markets, or conflicts. It belongs to the AI & ML Security category of cybersecurity.
What does AI-Generated Disinformation mean?
False or misleading content produced or amplified by generative AI to deceive audiences, manipulate opinion, or influence elections, markets, or conflicts.
How does AI-Generated Disinformation work?
AI-generated disinformation uses large language models, image generators, and voice or video synthesis to mass-produce convincing fake news, social-media posts, fabricated interviews, and synthetic evidence. Compared with traditional propaganda, generative AI lowers the cost of fluent multilingual content, enables on-demand personalisation per target audience, and allows fast iteration after fact-checks appear. Documented cases include AI-generated robocalls impersonating political candidates during the 2024 elections and synthetic videos used in influence operations around the war in Ukraine. Mitigations span content-provenance standards (C2PA), platform-level detection, watermarking of synthetic media, media-literacy education, and legal regimes such as the EU AI Act's transparency rules for deepfakes.
How do you defend against AI-Generated Disinformation?
Defences for AI-Generated Disinformation typically combine technical controls and operational practices, as detailed in the full definition above.
What are other names for AI-Generated Disinformation?
Common alternative names include: Synthetic disinformation, Generative AI propaganda.
● Related terms
- ai-security№ 1208
Voice Cloning Attack
An attack that uses AI-generated speech mimicking a real person to bypass voice authentication or trick victims into authorising payments or actions.
- ai-security№ 1203
Video Deepfake Attack
An attack that uses AI-generated synthetic video of a real person, often in a live meeting, to authorise fraudulent transactions or spread disinformation.
- ai-security№ 297
Deepfake
Synthetic audio, image, or video media generated by AI to convincingly depict a real person saying or doing something they did not.
- ai-security№ 037
AI-Generated Malware
Malicious code written, mutated, or assisted by large language models, lowering the skill bar for attackers and accelerating variant production.
- attacks№ 1065
Social Engineering
The psychological manipulation of people into performing actions or disclosing confidential information that benefits an attacker.
- ai-security№ 026
AI Content Detection
Tools and techniques that estimate whether a piece of text, image, audio, or video was produced by an AI model rather than a human.