The world is ablaze with artificial intelligence, and nowadays everywhere you turn you find articles about ChatGPT. There are those trying to spice up their cooking, those raising privacy concerns, and some who think their dog owes its life to it.
The GPT in the name of the service stands for Generative Pre-trained Transformer, and the term refers to a large language model created using deep learning and refined with human feedback, which can perform autonomous text generation tasks based on user-specified commands. ChatGPT is the name of the interface for interacting with version 3 of the language model using dialog. This service is available free of charge to anyone who registers, and can be used in several languages.
However, not only can you use it in multiple languages, but it will also translate the text you type into the language of your choice. This has immediately caught the attention of the translation industry. Some expect it to change the industry radically, just as machine translation did in the past, while others are trying to capitalize on the hype. After all, if you could triple the stock market value of a company with a name change in the heyday of blockchain technology, it might be worth trying to sell a service for translating ChatGPT-generated text.
Setting these special cases aside, serious industry players have also shown interest in the translation capabilities of the new model, and have conducted their own tests. The different methodologies agree on testing languages with large corpora and less common language pairs separately.
A study conducted by Microsoft compared ChatGPT and other GPT models to leading scientific and business neural machine translation (NMT) systems. The different solutions were tested at both sentence and document level, and subjected to both automatic and human evaluation. Out of the 18 language pairs tested, the GPT 3.5 system (which is newer than ChatGPT) outperformed the standard NMT solutions for the German-English, Japanese-English and Chinese-English pairs, and performed similarly to NMT systems for the other language pairs. The researchers concluded that for languages with a large corpus, the new system can produce competitive translations, but in the case of complex sentences, it often came up with unnatural solutions, so it may be less suitable for translating technical texts.
Intento’s research worked with a much more limited dataset, but also looked at the capabilities of GPT 3.5. They compared the large language model’s capabilities to popular NMT models (e.g. Google, DeepL, and Microsoft) in the English-Spanish and English-German language pairs. In the English-Spanish language direction, performance for general texts was almost identical, but English-German GPT translations were only sufficient to reach the mid-range. For medical and legal texts, GPT 3.5 performed significantly worse than the neural machine translation systems in both language pairs. Another important difference is that GPT’s system took 3-4 seconds per segment to translate, which is about 10 times slower than NMT systems.
The neural machine translation service provider Tencent has also conducted a study on the subject, comparing the ChatGPT system (which, unlike the subject of the previous studies, uses GPT version 3) with Google, DeepL, and its own NMT system. In this case, the languages used were English, German, Romanian, and Chinese. Similar to Microsoft’s results, the GPT model performed close to neural systems in the case of general topics and language pairs with rich corpora (e.g. English-German), while it performed significantly worse for language pairs relying on a smaller text base. The gap between systems was even wider when it came to specialized topics such as life science texts.
As the above research shows, the emergence of ChatGPT and the underlying GPT models has not yet resulted in a radical change in the daily lives of translators, although it is certainly convincing that it uses a different approach that comes close to the performance of existing neural machine translation engines, even if it works much more slowly. And given that NMT engines are already a relatively mature technology and the capabilities of generative models have recently been improving rapidly, it is conceivable that the latter will overtake their competitors within a few years.
Although ChatGPT should not yet be considered as a new source of material for post-editing, it may have a place in the daily lives of translators, as it can help with certain translation tasks. For example, it could be asked to shorten a text (even to a precise number of characters), help with research by summarizing longer materials, or even provide stylistic assistance by rephrasing the target sentence in a different register.
At this point, it is important to note that the data entered in the free ChatGPT platform are being used to further refine the model, so the system is obviously not intended to be used to support commercial translation projects. The paid versions, however, guarantee data security and offer an application programming interface (API), so they could be suitable for business use in the long term.
We didn’t ask for help from the AI to write this article, but in researching the topic, we came across more than one article that felt like it was written by ChatGPT. This is a good reminder that, if not directly in translation, some radical changes can be expected in the world of content production in the near future.
