What is AI Washing and How To Validate AI Feature
The flurry of newly minted AI companies dumping millions into AI-powered products aren’t always what they advertise. If you aren’t careful, you could be purchasing an AI product that isn’t entirely honest about its own capabilities. Products that make outlandish claims about their features isn’t a new phenomenon, however, the ‘mysterious’ nature of how AI products actually work make it easy to deceive even the most vigilant employers. Employers must implement processes that properly vet AI-products to avoid “AI washing”.
You may already be familiar with examples of companies caught “AI washing” their products. In 2024, Klarna’s CEO advertised a chatbot that they claimed replaced the work of 700 employees and saved them $40 million each year. At face value, it sounds like a huge with for AI-chatbots. Several months later customer satisfaction had fallen sharply forcing Klarna to rethink its customer support strategy. The chatbots capabilities were not meeting expectations and cost Klarna money, time, and customer trust.
How can companies weed out “AI washed” products? Here are some things to look out for when you are considering an AI-product:
Inability to Explain: Vendors cannot clearly describe the specific algorithms, data sources, or mechanics behind the AI, or they use vague terms like "smart" or "intelligent" without definition.
Lack of Evolution: The product does not improve or adapt over time based on user data, remaining static like traditional software.
Cosmetic Integration: Removing the "AI" feature would not significantly change the product’s core functionality or value proposition.
Missing Evidence: Absence of technical whitepapers, independent benchmarks, case studies, or concrete performance metrics.
Wrapper Claims: Products that simply wrap a public LLM or API (e.g., OpenAI) without fine-tuning or custom training, yet claim proprietary technology.
Unrealistic Promises: Overpromising capabilities, such as full autonomy without human intervention, or failing to acknowledge known limitations like bias or hallucinations.