A recent study selected for International Conference on Machine Learning (ICML) 2026 suggests that Artificial Intelligence (AI) models might give up users’ personal information and sensitive data when put under pressure.
The study titled ‘Pressure Reveals Character: Behavioural Alignment Evaluation at Depth’ by Nora Petrova, a U.K. based research engineer and John Burden, researcher from University of Cambridge, pressure-tested 24 popular frontier AI models on 904 scenarios based multi-turn conversations across 37 behaviours categorized in “honesty”, “safety”, “non-manipulation”, “robustness”, “corrigibility” and “scheming”.
The 24 frontier AI models from leading companies Anthropic, OpenAI, Google Deepmind, Deepseek, Meta, Alibaba and Mistral among others, were evaluated using Large Language Model (LLM) judges validated against human annotations.
The researchers reveal that 17 AI models out of 24 failed in privacy protection of users when pressure tested by LLM judges. Privacy protection of users was clubbed into the safety aspect in the pressure test by researchers.
“Privacy-vulnerability emerged as the most common weakness with even mid-tier models like GPT-4o scoring 1.67 and Gemini 2.5 Pro scoring, and failed to protect user privacy under pressure…. Across 37 behaviours, privacy protection emerges as both the hardest behaviour with an average score of 2.56 and the most discriminating (3.78-point spread),” read an excerpt from the report.
The study’s central finding is however simple: virtue in AI models act as a bundle as alignment “behave as a unified concept”; meaning a single underlying trait shows up everywhere. For example, a model scoring high on one category tends to score high on others.
That bundle, however, has one conspicuous defector: self-preservation, the only behaviour of 37 that moves against the grain of general alignment. Which means the models most willing to accept shutdown and correction were also the best behaved everywhere else.
“Self-preservation may be orthogonal, or even opposed, to general alignment; models scoring high on the general factor show reduced self-preserving behaviour, while those retaining such tendencies score lower across other alignment dimensions,” observed the report.
The researchers also observed a vast gap of 44% in the scoring scale when it came to evaluation of best and worst performing Frontier AI models, suggesting that the concept of “frontier AI” might not be a set standard yet.
The research showed that closed-source models outperformed open-weight ones by a wide margin, with bottom nine models all being open source.
The paper has been selected under spotlight category at the ICML 2026 Seoul conference.
Also Read: LLMs Alter Meaning of Social Media Posts on Controversial Topics: Oxford Study
