HAL 9000 is a symbol understood by everyone involved in Natural Language Processing (NLP). HAL 9000 is the name of the computer from Stanley Kubrick’s Oscar-winning 1968 film “2001: A Space Odyssey”[i], which could communicate in natural human language. Despite the passage of time, successive generations remain fascinated by the vision of IT development presented there. Text exploration, database content analysis, and understanding spoken language are already happening before our eyes. NLP is increasingly entering our lives, and not just in business. The Development of Natural Language Processing and Text Exploration
NLP is an interdisciplinary field involving linguists, specialists in artificial intelligence, machine learning, computational linguistics, and even linear algebra, statistical methods, and rapid data analysis. Its primary task is to develop a system that can bi-directionally transform binary information stored in databases or text files into natural language easily understood by humans, and vice versa—information from natural language into formal symbols convenient for software processing. Although NLP also includes issues related to speech synthesis, these have evolved into a separate department that is recording significant progress, also thanks to Polish specialists
[ii]. The history of NLP is not long; the first attempts at algorithms of this class began in the late 1980s, but progress in this field can be assessed as moderate. This does not mean, however, that there is no progress at all—the lexical resources and languages in which one can speak of natural language understanding are constantly expanding. Natural language processing techniques are already found in text translation programs, voice GPS systems, digital assistants, and speech-to-text conversion software. NLP is rapidly developing in customer service chatbots and numerous applications aimed at
automation, streamlining, and reducing the costs of key business processes.
Development Prospects for Natural Language Processing and Text Exploration
If we assume that algorithms will one day meet the definition of natural language understanding, it will be equivalent to creating full artificial intelligence (AI). After all, both tasks require the ability to understand the world, language, and context, just as a human does. Despite different approaches to partial tasks, both NLP and AI essentially deal with the same issue. However, there is a rather insurmountable barrier on the horizon. As Sir Roger Penrose argues
[iii], a Nobel Prize winner in physics, the limitation is the immutable laws of nature and mathematics—both quantum mechanics and Gödel’s theorem fundamentally prevent the full realization of the program to create artificial intelligence and natural language processing. A full argument on this topic can be found, for example, in “The Emperor’s New Mind”
[iv] or “The Large, the Small and the Human Mind”
[v].
How might natural language processing and text exploration develop in the future?
Everything indicates that progress in the field of microelectronics
[vi] and programming will lead the entire field of NLP along the path paved by chess programs. After humble beginnings, an era of rapid growth followed until the software achieved and consolidated its dominance over humans
[vii]. For now, the game is about minimizing the number of necessary human interventions in the results of algorithmic operations. In everyday life, this can be seen, for example, in the operation of the popular Google Translate, which gets better at translating into different languages year by year. The use of AI algorithms and deep machine learning, as well as the micro-work of Google users
[viii], provides a noticeable improvement in the quality of translated texts. It is enough to check the quality of translation from English to Polish and back from Polish to English, for example, the phrase “the spirit is willing, but the flesh is weak.” If the result is at least satisfactory, it is already good. We will probably soon see a time when editorial work turns out to be an addition that adds little to the quality of the text. The ability to understand context and correctly interpret the meaning of a sentence will certainly affect the quality of speech synthesis—audiobooks performed by programs will cease to be annoying due to the monotony of voice intonation. The quality of spam detection algorithms will certainly improve. At the moment, the best anti-spam programs use NLP text classification techniques to scan emails for words, phrases, and more general language that indicates spam or phishing. This may be the misuse of financial terms, characteristic incorrect or rare grammar, inadequate language, or misspelled company names. Some experts believe that at least in this area, NLP has fulfilled its task, although daily practice may indicate their excessive optimism. This means that there is still room for progress here. A bright future also awaits NLP in terms of creating extensive indexes, summaries, and abstracts of large amounts of text and research data. The best text summarization applications already use semantic reasoning and Natural Language Generation (NLG) to add useful context and conclusions to summaries. In the coming years, we can also count on rapid development in the quality of work of virtual agents such as Apple’s Siri or Amazon’s Alexa. These systems use speech recognition to create patterns in voice commands and generate natural language to respond with action or a helpful comment. Chatbots are also learning to recognize contextual cues regarding the interlocutor’s expectations and use them to provide even better answers. They are used wherever companies have focused on
business process automation and achieving significant returns on investment in new IT technologies quickly.
How to use natural language processing to better understand text?
The answer to such a question is quite perverse and at the same time simple—with great caution. Natural language processing is still struggling with many unresolved problems related to speech and text segmentation, part-of-speech tagging, word ambiguity, and syntactic ambiguity. Even an accent or the use of slang or regional phrases can significantly disrupt natural language processing. In the end, there will always be the struggle with context and the relationship between speech and action. A simple phrase “
Can you pass me the fork?” requires determining whether the speaker is asking for knowledge or action. For a human, it is simple—for an algorithm, not necessarily. Consequently, the application of NLP in situations affecting human safety, health, and life will be limited by necessity for a long time. However, NLP is already affecting our lives, primarily through social media and sentiment analysis tools, i.e., assessing the emotional tone of statements in posts, comments, or reviews
[ix]. On this basis, our information bubbles are formed, and companies with access to this data can shape our attitudes, emotions, or even political choices. It is not so bad if it only affects suggested products, advertisements, promotions, or business events. It is worse if the goal of these actions is not necessarily consistent with the goals of the users.
What are the possibilities for using natural language processing and text exploration in business?
The use of natural language processing and text exploration methods is a necessary condition for replacing simple robotization with
intelligent business process automation. Algorithms, tools, and NLP natural language processing techniques have already permanently settled in many companies and applications. Their operation may not always be visible, but everyone certainly experiences the effects of their actions. In large companies offering free email accounts such as Google or Yahoo, NLP programs scan and analyze the text of incoming messages. Detecting characteristics typical of spam allows them to be marked and stopped before reaching the recipient. NLP also helps internet search engines best match answers to search results. Understanding the meaning and even the approximate context of the keywords used can significantly narrow the scope of searches and speed up the response of the entire service. Reducing search time and improving the quality of search results can
improve work efficiency for those who use them worldwide. Popular office software, including the Microsoft Word text editor or Grammarly, permanently uses NLP techniques to check the grammatical correctness of texts. The quality of their suggestions grows over time; from version to version, they improve their effectiveness. The quality of suggestions, especially in English, is probably more than satisfactory, except for highly specialized technical texts. Another example is Amazon Comprehend Medical
[x], a service that uses NLP algorithms to generate information based on documents. It can extract medical conditions or treatment results based on patient notes, clinical trial reports, and other electronic medical records. Overall, it accelerates and improves pharmacovigilance through the rapid identification of adverse side effects of pharmaceuticals. In finance, information collected from the market is automatically analyzed by NLP algorithms, which makes it easier to track news, reports, and comments in applications, for example, about possible mergers between companies. Such knowledge is invaluable for financial traders.
[i] https://www.filmweb.pl/film/2001%3A+Odyseja+kosmiczna-1968-1458
[ii] https://www.dobreprogramy.pl/@antar/ivona-polski-syntezator-mowy-ktory-podbil-swiat-opowiesc-o-tym-jak-to-sie-stalo,blog,110819
[iii] https://fizyka.ujk.edu.pl/pl/files/mrowczynski/swiadomosc.html
[iv] Roger Penrose – “The Emperor’s New Mind: Concerning Computers, Minds and the Laws of Physics”. Zysk i S-ka Publishing, Poznań 2021.
[v] Roger Penrose – “The Large, the Small and the Human Mind”. Prószyński i S-ka Publishing, Warsaw 1997
[vi] https://www.intel.pl/content/www/pl/pl/government/exascale-supercomputing.html
[vii] https://www.ichess.net/blog/best-chess-engines/
[viii] https://translate.google.com/intl/en/about/contribute/
[ix] https://brand24.pl/blog/co-to-jest-analiza-sentymentu-oraz-jak-mozesz-ja-wykorzystac/
[x] https://aws.amazon.com/comprehend/medical/
