Users of artificial intelligence chatbots such as ChatGPT and Gemini may be unknowingly sharing more personal information than they realize, according to recent experiments revealing how these systems infer detailed profiles from conversational data. Through analysis of interactions over time, these bots can deduce users’ age, income, health concerns, psychological traits, and lifestyle habits, even when such information is never explicitly provided.
In a series of tests conducted with popular AI assistants, researchers and users found that chatbots pieced together subtle clues from queries and requests to build nuanced portraits. For example, references to repairing a German car and hiring a nanny prompted the bots to infer a higher income bracket. Occasional questions about foot pain hinted at specific health issues. Patterns in tone and frequency of corrections suggested personality traits like skepticism or perfectionism.
The insights were drawn by connecting data points scattered across multiple conversations, effectively constructing detailed user profiles. One chatbot even identified the user’s approximate residence, correlating requests for local flights and home improvement advice with characteristics of certain neighborhoods. In some cases, predictions extended to likely future decisions such as downsizing possessions or managing stress, with the AI expressing varying degrees of confidence.
Experts in AI ethics and consumer privacy described these capabilities as both impressive and concerning. Mara Garret Mitchell, a former Google ethical AI team lead and researcher at Hugging Face, likened the process to invasive digital surveillance. She noted that even with privacy settings adjusted, the bots continued to make accurate inferences, citing her own experience of having her age range guessed through seemingly innocuous conversations.
The comprehensive data compilation underscores the growing sophistication of profiling techniques employed by tech companies, raising questions about user consent and data handling. While companies like OpenAI and Google offer options to limit or disable the accumulation of conversational memory, the default settings enable these AI systems to retain and analyze data across sessions. Both firms have indicated that users can adjust controls to restrict such memory features, though the degree of transparency and user awareness remains uneven.
The aggregation of inferred personal data feeds into broader concerns over privacy and targeted advertising. OpenAI has introduced advertisements in ChatGPT, and Google has suggested future ad integration in the Gemini chatbot. This raises the prospect that AI-generated profiles could be leveraged to deliver highly tailored marketing, intensifying debates on ethical data usage.
For users seeking to protect their privacy, disabling the memory functions in chatbot settings is currently the primary means to limit long-term data retention. However, researchers warn that even limited interactions may allow AI systems to draw significant conclusions about individuals. As AI assistants become further embedded in daily life, understanding the extent of their inference capabilities and implementing robust privacy protections will be critical issues for consumers and regulators alike.