Google Blocks Large-Scale AI Model Cloning Attempt

Google’s Threat Intelligence Group recently identified and blocked a major campaign involving over 100,000 prompts, which the company asserts were designed to duplicate the proprietary reasoning capabilities of its Gemini AI model. This effort, detailed in a recent quarterly threat report, represents a sophisticated attempt at what is known as model extraction or distillation. This machine-learning process involves creating a smaller model that mimics the essential characteristics of a much larger, more complex one.

The company’s advanced systems detected these prompts in real time, effectively mitigating the risk of this particular attack. By protecting its internal reasoning traces, Google prevented potential intellectual property theft. The technology giant is actively working to prevent competitors from leveraging its substantial investments in AI model development to train their own systems.

Protecting Proprietary AI Capabilities

Google views these extraction attempts as a significant form of intellectual property theft. The company stated that such activity allows attackers to rapidly accelerate AI model development at a drastically reduced cost. This poses a direct threat to the considerable resources and innovation Google has poured into its AI research.

In the specific campaign Google identified, attackers instructed Gemini to maintain “the language used in the thinking content strictly consistent with the main language of the user input.” This technique, according to Google, aims to extract the model’s reasoning processes across multiple languages. The wide array of questions suggested an intention to replicate Gemini’s reasoning abilities in various non-English languages across a broad spectrum of tasks.

Google consistently observes attempts at model extraction from both private sector entities globally and researchers seeking to clone proprietary AI capabilities. The company maintains that these attacks violate its terms of service and could lead to takedowns and legal action. However, legitimate researchers and potential customers might also seek large samples of Gemini’s reasoning for valid purposes, such as comparing model performance or evaluating its suitability for specific tasks before making a purchase.

Escalating Threats to AI Intellectual Property

The concern over model extraction is not exclusive to Google. OpenAI, another leading AI developer, recently informed United States lawmakers about similar activities. OpenAI alleged that the Chinese AI firm DeepSeek has employed “new, obfuscated methods” to extract results from prominent American AI models. The intention, according to OpenAI, is to train DeepSeek’s own systems, illustrating a growing worry among companies that have invested billions in AI development.

Ross Filipek, CISO at Corsica Technologies, noted a shift in cybersecurity threat priorities. He explained that model-extraction attacks do not involve traditional system infiltration but instead focus on transferring developed knowledge from a victim’s AI model. This knowledge is then used to expedite the development of the attacker’s own AI models.

Organizations that offer AI models as services should be highly concerned about the threat of intellectual property theft through model extraction. Google’s report advises these entities to closely monitor API access patterns for any indications of systematic extraction. Filipek emphasized that effective defense against these attacks requires strict governance over AI systems and meticulous monitoring of data flows. He also recommended that organizations implement response filtering and output controls to prevent attackers from discerning model behavior, especially in the event of a breach.

Nation-State Actors and AI Misuse

Google’s report also shed light on how government-backed threat actors are integrating AI models, including Gemini, into their operations. The company documented instances where state-sponsored groups from China, Iran, North Korea, and Russia incorporated Gemini into their activities in late 2023. Google promptly disabled accounts and assets linked to these malicious entities.

The Iranian threat actor APT42, for instance, reportedly utilized Gemini to craft targeted social engineering campaigns. The group fed biographical details of specific targets into the AI to generate conversation starters designed to build trust. Furthermore, APT42 employed Gemini for translation purposes and to comprehend cultural references in non-native languages, enhancing the sophistication of their deception.

Chinese groups APT31 and UNC795 leveraged Gemini to automate vulnerability analysis, debug malicious code, and research exploitation techniques. North Korean hackers from UNC2970 reportedly mined Gemini for intelligence on defense contractors and cybersecurity firms, gathering details on organizational structures and job roles to bolster their phishing campaigns. Google’s swift action involved disabling the associated accounts, and Google DeepMind utilized these insights to bolster its defenses against future misuse of its AI technologies.

AI Integration in Malware Operations

Beyond nation-state activities, Google has also observed other forms of Gemini misuse, including the direct embedding of its APIs into malicious code. The company identified a new malware family it named HONESTCUE, which integrates Gemini’s API directly into its operational framework. This malware sends prompts to the AI to generate working code, which it then compiles and executes in memory.

The prompts used by HONESTCUE appear benign in isolation, allowing them to circumvent Gemini’s safety filters, according to the report. Pete Luban, field CISO at AttackIQ, highlighted that public AI models like Google Gemini offer hackers immediate access to powerful large language model capabilities without requiring them to build or train anything from scratch. This instant access significantly enhances malware capabilities, enabling faster lateral movement, stealthier attack campaigns, and more convincing mimicry of typical company operations.

Google’s report also documented COINBAIT, a phishing kit constructed using AI code generation platforms. Another service, Xanthorox, advertised custom malware-generating AI but was actually a wrapper around commercial products, including Gemini. Google promptly shut down accounts and projects connected to both COINBAIT and Xanthorox. Luban underscored that the rapid evolution of AI-enabled threats renders traditional defenses insufficient. He stressed the importance of continuous testing against realistic adversary behavior to ensure that security defenses are adequately prepared to combat these adaptive threats.