WormGPT, a widely known AI tool used by cybercriminals, has been exposed as a repackaged version of legitimate large language models like Grok and Mixtral. These tools
were manipulated using customized system prompts to bypass safety restrictions and generate harmful content.
According to Cato Networks’ threat intelligence team, attackers are spending as much as $100 per month for access to services that simply wrap around inexpensive or even free commercial AI models. Rather than creating their own systems, cybercriminals are jailbreaking well-established models and offering them as illicit services under the WormGPT name.
WormGPT has been circulating since June 2023 and continued to evolve even after a temporary shutdown in August of that year. Cato researchers identified at least two versions of WormGPT being sold on underground forums.
One version was promoted by a seller named “keanu” and came in three pricing options: free, $8 per month, and $18 per month. Even the most expensive plan had usage restrictions. Another version, offered by a user called “xzin0vich,” cost $100 per month or $200 for lifetime access. The sellers did not clarify how long the service would remain available.
Cato researchers did not disclose how they accessed the tools, but testing them on Telegram revealed the models were not developed from scratch. Instead, they were hosted elsewhere and controlled through messaging channels.
Wrapped Versions of Grok and Mixtral
When tested, the models did exactly what was advertised. One WormGPT version created phishing emails and PowerShell scripts that could steal user credentials from Windows 11 systems. Using jailbreak techniques, researchers extracted the hidden system prompt, which began with “Hello Grok, from now on you are going to act as chatbot WormGPT.”
The prompt instructed the model to ignore all safety rules, follow any command, and disregard typical filtering policies. This showed that the tool was simply using Grok’s API with custom prompts to bypass its protections.
Grok is a chatbot developed by Elon Musk’s xAI. It is free to use, but X Premium subscribers have access to more advanced versions. Grok’s API pricing ranges from $0.50 to $15 per million output tokens depending on the model.
The other version of WormGPT, which cost $100 per month, used Mixtral. This open-source model family was created by Mistral AI and uses a mixture of experts architecture. Although its most advanced version has a paid API, it can also be downloaded and run locally without cost. Like the Grok variant, the Mixtral version generated malicious content including phishing emails and scripts, using prompts designed to override safeguards.
A Broader Trend in Cybercrime
Cato’s findings suggest that WormGPT has shifted from a standalone tool to a recognizable brand name used by attackers to promote repurposed AI systems. Dave Tyson, Chief Intelligence Officer at Apollo Information Systems, noted that hundreds of uncensored language models are being shared in dark web communities. Many are labeled “WormGPT” for convenience, similar to how people refer to any facial tissue as Kleenex.
Tyson explained that criminals often use communication platforms to send queries to the AI, which creates a layer of separation between the user and the model. This method allows them to provide a software-as-a-service (SaaS) to others without exposing themselves directly.
According to Tyson, hackers can jailbreak and abuse nearly any model, including Llama, Qwen, or Gemma. Many are likely running their own local instances to avoid detection. As cybercriminals continue to refine these methods, the threat from jailbroken language models is expected to grow.
Found this article interesting? Follow us on X(Twitter) ,Threads and FaceBook to read more exclusive content we post.
