China Prosecutes Over 190 People for AI-Generated Misinformation and Fraud

How AI-Based Fraud Works

According to Jiang Guoli, Director of the Criminal Investigation Bureau at China's Ministry of Public Security, criminals are increasingly leveraging AI and emerging technologies to commit telecom fraud, online scams, privacy violations, undermine public trust, and in some cases threaten national security.

Common Tactics

1. Deepfake Impersonation

• Criminals use AI-generated images, audio, and video to impersonate acquaintances, friends, or authority figures
• They build trust before requesting money or sensitive information
• Scammers may repeatedly request facial photos to refine their deepfake models

2. Misinformation Campaigns

• AI generation tools are used to create and spread false rumors and fabricated content online
• Such campaigns disrupt public order and sow confusion

3. System Infiltration and Data Theft

• AI is used to breach government and business information systems
• Personal data and assets are stolen

Warning Signs to Watch For

• Frequent video calls from unknown accounts
• Calls with recorded or artificial-sounding audio
• Repeated requests for facial photographs or biometric data
• Video call requests from overseas numbers
• Sudden financial requests from known contacts
• Subtle anomalies in voice, speech patterns, or facial movements

Prevention and Protection Steps

1. Verify Identity Independently: When receiving financial requests, always verify the requester's identity through alternative channels—direct contact, official phone numbers, or in-person confirmation
2. Protect Biometric Information: Be cautious about sharing facial images, voice recordings, video calls, or location data
3. Use Verification Tools: China's National Anti-Fraud Center app allows users to upload suspicious videos, audio, and images to detect AI-generated content and manipulation
4. Don't Engage with Suspicious Requests: Decline unusual requests and report them immediately to authorities
5. Be Alert to Speech and Visual Inconsistencies: Pay attention to unnatural speech patterns, lip-sync issues, or subtle facial movements

Official Response Measures

Chinese authorities have implemented several countermeasures:

• Big data-driven risk assessment models to identify suspicious activity
• Integration of AI detection functions into anti-fraud apps
• Regulatory strengthening of AI services (mandatory content labeling)
• Over 600 inspections and more than 30 administrative actions taken in the first half of 2024
• Collaboration on legislation against cybercrime and safety assessment mechanisms

How to Report

If you encounter suspicious communications or suspect fraud, report immediately to local police departments or the National Anti-Fraud Center.