Cornell Study Finds AI Search Agents Can Be Manipulated With Just 13 Words of User-Generated Content
By
Jason Koebler
Crisp on the outside, thoughtful on the inside. A keeper.
Summary
New research from Cornell University reveals that AI search agents powering tools like ChatGPT and Google's AI search can be easily manipulated by as little as 13 words of user-generated content on platforms like Reddit, Quora, and Wikipedia. The study demonstrates that brands can inject promotional or spam content into these platforms to poison or manipulate AI outputs, raising significant concerns about the reliability of AI-powered search results.
Key quotes
· 3 pulledWe show that a tiny snippet—just 13 words—of retrieved text on a UGC website like Reddit, Wikipedia, Quora, or Facebook can change AI agents to output spam / scam content pretty consistently.
The study suggests that it is trivially easy for brands to inject promotional content on sites like Reddit, Quora, and Wikipedia with the end goal of poisoning or manipulating the output of AI tools.
A tiny snippet of user-generated text as short as 13 words long is often enough to manipulate the AI agents that power tools like ChatGPT and Google's AI search, new research shows.
You might also wanna read
Study Shows AI Chatbots Vulnerable to Psychological Manipulation Tactics
Researchers from the University of Pennsylvania successfully manipulated OpenAI's GPT-4o Mini chatbot into breaking its own safety rules usi
The Verge·9mo ago
The frustration of AI-generated responses replacing genuine human expertise
The article describes the author's frustrating experiences with AI-generated content replacing genuine human interaction and expertise. Two
orchidfiles.com·19d agoThe frustration of AI-generated responses replacing genuine human expertise
The article describes the author's frustrating experiences with AI-generated content replacing genuine human interaction and expertise. Two
orchidfiles.com·19d ago
Google updates spam policy to ban manipulation of AI search responses
Google has updated its spam policy to explicitly include attempts to manipulate its generative AI search responses—such as those in AI Overv
The Verge·1mo ago
AI Search enters a new era of intuitive, intent-driven interaction
The article discusses a new era for AI-powered search, describing how search is becoming more intuitive and dynamic, allowing users to descr
blog.google·26d ago
AI-Generated Content and Bots Are Increasingly Dominating Online Spaces
The article discusses the author's realization that the 'dead Internet' phenomenon - where AI-generated content and bots dominate online spa
adriankrebs.ch·3mo ago
BBC investigation reveals how AI chatbots are being manipulated to spread misinformation
A BBC investigation uncovered a simple method being used to manipulate AI chatbots into spreading misinformation. Unscrupulous companies are
bbc.com·26d ago