Recognizing whether new AI tools will actually automate your processes or are merely a costly illusion known as fauxtomation has become a key competency for decision-makers today. Effective artificial intelligence in business must generate a measurable return on investment, rather than just risky slop or hallucinations that damage brand reputation. In this article, we analyze how to critically verify vendor claims, improve the quality of language model responses, and select KPIs that confirm the real value of implemented technologies.Everything is “AI” these days
As stated in the report published by IAB Poland, “
Artificial Intelligence Guide“, there is now an application for AI in practically every industry. In e-commerce, it is used for customer segmentation and product description generation; it assists copywriters and marketers in content creation, and in the manufacturing industry, it is used to automate production lines. Interest in AI is also confirmed by market size – in 2024 alone, the
AI industry was valued at $184 billion, and it is expected to reach $826 billion by 2030! Almost everyone knows Sam Altman, CEO of OpenAI, and ChatGPT, his organization’s flagship product, is used by millions of people worldwide. But it is worth asking: are all AI applications justified? Are the voices of those skeptical about the AI boom being heard correctly? It seems that AI is popping up everywhere (literally—Samsung has introduced
AI-powered refrigerators for creating recipes and shopping lists), and investors seem interested in the development of this technology, but does it always make sense? Considering some AI applications, one might think of the term “
fauxtomation“, or fake automation. In other words, this refers to automation that doesn’t actually automate anything. There are many examples –
Chirper is a social platform populated only by AI bots, the
AI Excuse Generator creates excuses, and
Grover is an AI for generating fake news. All these solutions demonstrate the power inherent in artificial intelligence, but in their case, the question “why?” is more than justified.
Does connecting GPT to a project make it an “AI” revolution?
The next question is: if an organization integrates ChatGPT (or another large language model) into its workflow, is it participating in the AI revolution? The short answer is “yes and no,” but it definitely requires elaboration. First, consider the arguments in favor of such a solution. There is no denying that GPT has immense capabilities, and many examples can be found in marketing—AI is integrated with Google Forms or Google Sheets and helps organize emails in Microsoft Outlook. It facilitates information retrieval—Bing has been integrated with ChatGPT since 2023, and Google is slowly introducing its own solutions in this area. It seems that integrating various tools with GPT offers only benefits, but a closer look shows that the image of AI is not as crystal clear as its creators would like. AIs have a tendency to fabricate data (which AI specialists diplomatically call “
hallucinations“), which highlights their dependence on the quality of training data. Ignoring this problem can lead to the situation Google found itself in mid-2024—because the AI built into the search engine treated the satirical site The Onion as a valuable source of knowledge, it encouraged users to
add rocks to their diet. In other words, connecting GPT to company tools will be useless if it is not preceded by an analysis of needs and capabilities. Various AI solutions have proven their usefulness many times, but applying them without thought can lead to many problems.
How to verify the actual use of AI algorithms?
Today, companies are wondering not only if they can afford to
maintain their own language models, but also how to verify the use of AI algorithms. AI can significantly improve work efficiency, but without proper evaluation methods, it can degrade quality. The best example is online content creation—yes, AI has allowed for the generation of large amounts of material in a short time, but it has also led to the “cluttering” of the web. The level of this clutter (e.g., through AI-generated images) is already so high that SEO experts say we used to have spam, and today we have
slop, which is a mass of texts and images that appear credible but turn out to be false upon closer inspection. However, there are several methods for verifying GPT or other AI tools. For proprietary models, it is worth ensuring the appropriate quality of training data—the better it is (i.e., properly cleaned, prepared for formats, and free of unconscious biases), the better the model’s output will be. The value of AI tools can also be verified based on daily work with them—in this case, the ability to prompt correctly is particularly useful. Prompts (commands) should be written as detailed as possible—it is good to include the context in which the result will be used, provide examples and sources, describe step-by-step what the machine should do, and set constraints (e.g., instructing it not to generate responses in table format). This way, the quality of responses can be improved.
The short answer to the question in the headline is: if a given AI solution fulfills its tasks and does not generate costs exceeding the company’s financial capabilities, it serves its purpose. However, it is worth reflecting for a moment on the value of AI tools, as there are many ways to evaluate it. The basic method for evaluating GPT plugins or other AI tools is to assess their alignment with the company’s business goals. Clearly defining requirements for an AI solution and the goals it is meant to achieve greatly helps in assessing its effectiveness. If, for example, AI is to be used to automate a production line, one should check how its implementation translated into production volume and how it reduced costs. Here, we can move to the second method—measuring the effects of AI solutions. To do this, specific KPIs must be established and consistently applied. If a GPT plugin is meant to help optimize a company’s marketing activities, it is good to measure how its use translates into conversion. If it is used for data analysis, it is worth checking how much faster the analysis process becomes. The value of AI tools can also be assessed by asking their users for their opinion. Since they will most likely use them daily, they will have the best idea of what works well and what needs improvement. It is also worth remembering that AI solutions are not a magic spirit that will transform the face of a company upon arrival. It is worth approaching them just like other business tools—critically—and optimizing their functions based on their performance results. Above all, however, do not forget that AI is just a tool and needs a human to function, and its effects should be useful, especially for people.
