An AI-enabled pointer (graphical icon on computer screen) that understands what a user points at is now live in Chrome and the feature will ship with Google’s Googlebook laptops. Google Deepmind made the announcement on May 12 through a research blog authored by researchers Adrien Baranes and Rob Marchant.
According to Google Deepmind, the pointer is powered by Gemini and is built around four principles: Keeping AI available across all applications without forcing users to switch windows, allowing users to point instead of writing detailed prompts, enabling shorthand commands such as “fix this” or “move that” and converting pixels into actionable entries such as dates, locations and objects.
Users can point at the content on a webpage and query Gemini about it without typing a prompt. The feature called Magic Pointer, will ship with its Googlebook laptop line later this year, in partnership with Acer, Asus, Dell, HP and Lenovo, said the company. Experimental demos are available in Google AI studio.
The announcement landed the same day META employees distributed protest flyers at multiple US offices against the company’s mouse and keystroke surveillance program.
META’s Model Capability Initiative captures mouse movements, keystrokes and screen snapshots on employees to train AI models. Employees argued the program may violate the National Labour Relations Act in the US and circulated a petition.
In response to queries from Reuters, META said the initiative exists because training AI agents require real examples of how people use computers, including mouse movements, button clicks and dropdown navigation.
The company said data would not be used for performance assessments and that safeguards protect sensitive content without specifying how sensitive content is defined.
CNBC reported that several Meta employees described the program as “dystopian” citing concerns about exposure of passwords, product details and workers personal information.
Also Read: Google’s New AI Tool Aims to Warn Cities About Flash Floods Before They Strike
