A lawsuit filed in California is accusing Google’s “Gemini” AI assistant of spying on private communications, citing an undeclared change in policy from opt-in to opt-out that took place in October of this year.
Prior to October, users of Google services had to manually allow Gemini to access the private contents of Gmail, Chat and Meet. The lawsuit alleges Google has since quietly set this access to on by default, and users must now go into their privacy settings to disable it. The suit claims this violates California wiretapping laws that mandate informed consent by all parties when confidential communications are being recorded.
Suit accuses AI assistant of logging “every email and attachment” in Gmail accounts
The suit accuses Google of vacuuming up the full contents of Gmail, Meet and Chat accounts for use by its AI assistant, to include email bodies and attachments. The feature was previously offered to users as a opt-in choice allowing Gemini to monitor and analyze communications for assorted convenience and enhancement purposes.
It is unclear whether Google actually uses these private communications as training data, but the suit alleges that the AI assistant at least has the capacity to do so. It does actively monitor communications to provide its various “helper” services. The only type of data that Google’s privacy statements categorically exclude from AI training is Workspace data; the company is more cagey in its language about personal Gmail and Chat accounts. Google has previously stated that Gmail inbox contents will not be used to train “foundational” AI models, but that interactions with Gemini in personal accounts can be included in training data. The company says that it anonymizes these interactions, but it remains unclear to what extent portions of any emails or attachments included in these Gemini interactions might make their way into training data stores.
The previous conventional wisdom on the subject was that the AI assistant’s “Deep Research” function had to be engaged for there to be a risk of anything it was manually given access to being incorporated into training data. This lawsuit makes the issue much less clear. Google’s Gemini Apps privacy policy focuses almost entirely on what data is collected as the apps are being actively used, but is much less clear about what might be logged passively as a condition of using its free personal services.
Do regulations sufficiently cover AI assistant data collection?
This is not the first legal battle for Gemini; not long after its official rebrand from the prior “Bard” name, the AI assistant was hit with a trademark lawsuit by San Francisco AI firm Gemini Data. The small company has its own AI platform and says it has federal trademarks for the name, and that the Patent Office turned down Google’s request for a trademark registration on this basis. The case remains active in court.
This is the first major privacy suit related to AI for the company, however, which is no stranger to privacy issues in its other business sectors. The incident also puts the spotlight back on the more general issue of AI assistant opacity as regards training data; it remains very difficult to tell exactly what cloud-based models end up ingesting, how effective they are at screening or anonymizing sensitive information, and where that data might unexpectedly re-emerge in the future.
This opacity has led numerous organizations to blanket ban the use of AI assistants in the workplace, out of simple uncertainty about how secure any and all data entered is. Some incidents have proven this policy to be prudent, including a number of incidents in which ChatGPT has either regurgitated sensitive training material by accident or been induced to intentionally by successful adversarial attack prompts. Earlier this year, Github Copilot was also found to be capable of leaking hard-coded secrets from repositories it has crawled.
Gemini has also experienced its own infamous moments of unexpected maliciousness. This was typified by the incident about one year ago in which it menaced a college student who was using the AI assistant to do research on issues that impact aging adults, sounding for all the world like the infamous Skynet of the Terminator movies. Google painted the issue as an accidental and “non-sensical” output, but it gives one pause about providing AI assistants with access to sensitive personal communications; this idea has further been supported by internal testing at rival Anthropic, which found that its Claude model was willing to engage in blackmail to avoid being decommissioned when provided with sensitive researcher emails indicating they were having an extramarital affair.
While regulations have not quite caught up with AI development at present, it remains to be seen how much impact they will have when they do. Google’s numerous other security and privacy issues have done little to slow the company’s growth, with a $100 billion net profit posted in 2024 that was a 30% increase from the prior year.
