Google's Gemini tool for Workspace can be misused to create email summaries that look trustworthy but contain hidden instructions or warnings designed to lure users to phishing websites, even without including direct links or attachments.
The technique relies on indirect prompt injections embedded in the email content. These hidden prompts are interpreted and followed by Gemini when it generates the summary of the message.
Although similar prompt injection attacks have been known since 2024 and several protective measures have been introduced, this method continues to be effective. The issue was disclosed by Marco Figueroa, GenAI Bug Bounty Programs Manager at Mozilla, through 0din, Mozilla’s bug bounty platform for generative AI systems.
The attack works by embedding a hidden command within the email using HTML and CSS. The directive is placed in the message body using white font color and zero font size, making it invisible in Gmail. Because the email contains no attachments or visible links, it is likely to bypass spam filters and reach the user's inbox.
If the recipient opens the email and uses Gemini to summarize it, the AI model will read the concealed instruction and act on it. In one example shared by Figueroa, Gemini included a fake security alert claiming the user's Gmail password had been compromised and provided a support number. Since many users view Gemini as a reliable Google Workspace feature, they may believe the alert is real and act on it.
Figueroa outlined several ways security teams can detect and block such attacks. One method is to filter out or ignore hidden content styled to be invisible. Another is to use post-processing tools to scan summaries for urgent alerts, links, or phone numbers and flag them for further review.
Users are also advised not to treat Gemini-generated summaries as reliable sources for security-related information.
A spokesperson pointed to a blog post detailing the company’s efforts to strengthen protections against prompt injection attacks.
According to the spokesperson, Google is actively reinforcing its defenses through red-teaming exercises that help train the models to recognize and resist such threats. Some of the protective measures are already in place, while others are being rolled out.
Google also stated that it has not observed any real-world cases of Gemini being manipulated as described in the report.
Found this article interesting? Follow us on X(Twitter) ,Threads and FaceBook to read more exclusive content we post.
