In this article, you will learn:
- What the limitations of traditional OCR technology are
- How GenAI can improve text recognition accuracy and reduce error rates
- Why well-crafted prompts are essential when working with GenAI and how they influence the accuracy of responses
- Why this developed solution can be successfully applied across various industries
Eliminate bottlenecks in process automation with a groundbreaking combination of GenAI technology and the UiPath platform, which is revolutionizing handwritten text recognition. Traditional OCR systems often fail with low-quality documents, generating errors that require costly validation; however, intelligent prompt engineering can reduce the error rate from 70% to just 20%. Learn how implementing advanced language models into document workflows helps offload staff and optimize business processes in the financial, retail, and administrative sectors. This universal solution marks a new stage in digital transformation, turning illegible notes into precise data ready for further processing.How to enhance RPA software capabilities for handling handwritten documents using GenAI
The solution developed by our experts using GenAI for text recognition has not yet been deployed in production, but it is already advanced enough to be presented to a wider audience. Especially since its application is not limited to a specific company or even a single industry. It is absolutely universal and will work well in retail, finance, and public institutions. One of the companies we work with provides its customers with the option to cancel services by sending a document containing basic customer data and specific service details. Such documents can be created using an electronic template that can be filled out on a computer, but they can also be filled out entirely by hand. A software robot retrieves documents sent to the company’s email address, then recognizes the text and classifies them into the appropriate group. At this stage, human validation is required in the UiPath Action Center module. This is how errors occurring during the text recognition phase are corrected. Finally, the validated data is entered by the robot into two different systems. The process is simple and the robot handles it excellently, but challenges arise at the OCR technology stage. The UiPath system recognizes the text and assigns appropriate tags to the document based on it. The robot receives the tagged documents. Thanks to this, it knows where the name, service details, and all other necessary information are located. However, the text itself often contains errors resulting from the difficulty of recognizing handwriting or mistakes, such as simple typos, made by the author. In this way, names like “Emikie” or “Dawyta” reach the robot. A human would likely guess immediately that it means “Emilia” and “Danuta,” but the robot would not. The number of documents containing errors reached 70%. This required a significant amount of work from people who validated the data entered into the systems by the robot. Although this seems like a lot, with such a large volume of data, automating the processing of even 30% of applications is considered a success. Nevertheless, appetite grows with eating. Especially since the engineers responsible for the robot’s work had several ideas for improving this result.
Searching for a solution
Therefore, the idea emerged to help the robot by using GenAI in the process. Its task would be to better understand the text and significantly increase the accuracy of text recognition, and consequently, reduce the amount of work that people would have to perform in the UiPath Action Center validation module. Initially, the use of the Levenshtein algorithm was considered, which could compare text with specified other texts and show how many characters would need to be changed to achieve compliance. Using this mechanism, one could choose the text most similar to the template. While this would be relatively simple and effective for individual data categories, it would not be for the entire dataset. The comparison operations performed would be counted in the millions for a single document. The use of Elastic Search software was also considered, but initial tests showed that it would not be possible to eliminate a significant number of errors. Finally, it was decided to test GenAI. This was simple because UiPath provides dedicated solutions allowing for the integration of artificial intelligence services. Ultimately, it was possible to reduce the number of documents with errors to 20%, which is a significant achievement compared to the starting point. Moreover, the solution is constantly being developed. Its creators hope that the number of errors will be even lower because artificial intelligence never stops learning.
How does it work?
The effectiveness of using GenAI, as in other cases, is based on an appropriate prompt. For each document field – first name, last name, service name, etc. – separate prompts were prepared. GenAI is asked to verify each field individually. It is essential to outline for the artificial intelligence the most detailed context possible in which it will look for answers. Asking a simple question like “what is this name” will not work. If only because the answer we receive will be too extensive. Meanwhile, we need an answer that is limited to indicating the correct name. Otherwise, processing the text generated by AI would be necessary. Therefore, the final prompt is more like information that we are providing the results of OCR software, informing that the result is not accurate and may contain errors, adding that we are looking for a Polish name, and that the answer should be limited to the name itself. Moreover, prompts can be enriched with indications of additional data sources. In the case of names, this could be a database of Polish names, and in the case of addresses, a GUS (Statistics Poland) database. You can also provide your own file, e.g., containing information about services or products. In this case, the ChatGPT service was used, but another language model could work just as well. Thanks to the integration tools prepared by UiPath, adding another service is very simple.
