In a report titled “Unlawful by Design: Exposing the Human Rights Costs of Generative AI”, Amnesty International, a global human rights organization, said leading AI models rely on what it describes as unlawful web-scraping practices, making them “unlawful by design” and posing risks to privacy, human rights and the environment.
“Companies across the world are supplying generative AI products under the veneer of efficiency and sophistication, but in reality, these systems perpetuate mass invasions of privacy through unlawful web scraping: an automated process for extracting data from websites, including personal data, such as images and social media activity, to train AI models,” said Likhita Banerji, Head of the Algorithmic Accountability Lab, Amnesty International.
“The extractive data pipeline, inherent design choices made by tech companies and exploitative supply chains, to build generative AI systems have enabled a paradigm of technology development that opens up a risk of mass abuse of human rights,” she added.
Amnesty International researched the models powering some of the most popular publicly available standalone generative AI tools, including GPT-3 by OpenAI, Google’s Gemini, Meta’s Llama, DeepSeek and tools by Midjourney and Stable Diffusion.
Generative AI doesn’t run on magic. It runs on massive data pipelines built on privacy violations by design.
Our new @Amnesty report exposes how big tech’s AI systems are powered by surveillance, data extraction, and abuse of people’s rights, at scale.
We researched the models…
— Erika Guevara Rosas (@ErikaGuevaraR) May 28, 2026
According to the report, such systems rely on extracting information from billions of public online posts and images, often without the explicit consent of the individuals appearing in or creating them.
The report further argued that not only do these systems infringe on privacy by design but, as AI models are trained on larger datasets, they also amplify hateful and discriminatory content along with negative stereotypes and prejudices, especially related to racial and gendered lines.
“Racial, gender and cultural biases are consistent features of generative AI systems, a product of the training data that is largely pulled from the web and therefore polluted with real-world biases which harm historically marginalized communities,” the report stated.
It further claimed that generative AI systems pose risks to the right to freedom of thought as they are capable of influencing users’ thoughts and shaping their personal beliefs through predictive suggestions. The report comes amid an ongoing global debate over the legality of using publicly available internet data to train AI models, an issue that remains under legal and regulatory scrutiny across multiple jurisdictions.
“These choices are not inevitable. We must challenge the design choices adopted by companies who build generative AI systems by relying on training data, including personal data, that is extracted non-consensually and on a grand scale,” Banerji stated.
“This is one of the most egregious practices among AI companies operating with disregard for human rights and must urgently be addressed. A different trajectory of technology development is possible if authorities act urgently to course correct,” she stressed.
Heavy Environmental Costs
The report raised environmental concerns due to the rapid expansion of generative AI companies.
“Generative AI production often results in a negative impact on communities that are historically marginalized as the lands and resources that belong to these communities are exploited to build data centres and fulfil processing requirements,” the report argued.
The report pointed to Google’s own sustainability report from 2024, according to which there has been a 48 percent increase in the company’s greenhouse gas emissions since 2019, attributable to data centre and supply chain emissions. Similarly, Microsoft’s emissions increased by 29 per cent between 2020 and 2024, attributable to data centres carrying out AI-supporting processes.
The intensive use of resources in generative AI production has led to communities from Cerrillos in Chile, and Querétaro in Mexico, to Arizona in the United States of America, resisting data centres in areas which are already affected by droughts and shortages in electricity.
As part of its research process, Amnesty International wrote to Google, OpenAI, Meta, Stability AI, Midjourney and DeepSeek, giving them an opportunity to respond to the findings of the research briefing, which Amnesty says found that their models are reliant on unlawful web scraping, among many other related human rights concerns.
The international organization also wrote to Intel and VMware specifically regarding the risks of discrimination, and to Google, Microsoft and Amazon about the environmental harms associated with their generative AI systems and related infrastructures. However, only Microsoft, Amazon, Intel, OpenAI and Meta responded.
Amnesty International is calling on states to prohibit standalone generative AI systems that have been built using what it defines as unlawful web scraping, defined as the bulk and mass collection of training data through the web.
It asked companies to immediately cease the practice of what it describes as unlawful non-consensual web scraping of personal data for AI training purposes, and stressed that states must hold companies accountable for human rights abuses linked to their design and business choices.
Also Read: Gig Workers Face Growing Algorithmic Control, Human Rights Watch Says
