Anyone who has in any way been involved in interpretation and translation understands the difficult nature of translating between two languages, considering all the different words, phrases, and most importantly the culture reservations in many cases. With that in mind, think of trying to invent a system that actually carries out all these tasks automatically. This is certainly a major challenge indeed. The reality is there are so many words out there, different phrases and also rules that cannot go neglected. The good news is that neural networks are able to take in huge and complicated sets of data like a piece of cake. Google has allocated a significant amount of assets on a technique involving machine learning translation, and after years of hard work they were finally able to officially debut it.
It is called the Google Neural Machine Translation (GNMT) system and it has been deployed to tackle Chinese-English queries, itself a very difficult task to accomplish. This is an upgrade in complexity from methods existing today. The nuts and bolts in short are as follows.
Methods of interpretation
A simple technique that can be used in translation–easy to comprehend and use by a small child or simple computer–would involve searching for the meaning of each word and replacing it with its counterpart in another language. However, we all know that speech nuances and very often the sheer meaning can be lost in such methods. However, such a rudimentary method under a word-by-word system can play a minimal role and act as a decent starting point.
Considering the fact that language is a naturally phrase-based matter, learning as many phrases and semi-formal rules possible would be the next logical step. And then comes the application of this earned knowledge. The thing is that a large amount of data and knowledge is required for such a feat, and not just a simple dictionary from one language to another. This involves a significant amount of statistical chops to understand the difference in various phrases. Take these three for example: “run a kilometer,” “run a check” and “run a shop.” This is a field of expertise where computers are very good at. Therefore, once they started taking over they helped transform phrase-based interpretation into the new norm, and never looked back.
To make things even more complicated is the concept of translating a complete sentence. This was another jump in sophistication, and of course, the computational power needed to realize such a necessity was also a significantly puzzling task. Enter neural networks with their expertise in devouring intricate rulesets and rendering a predictive model. Researchers have been probing into this subject and yet, Google has been able to provide this software faster than others. Simple as that.
GNMT can easily claim and boast of being by far the most advanced and effective tool to successfully take advantage of the power of machine learning in the field of interpretation. This technology takes a look at the entire sentence, while also taking into consideration–if we can actually use such a term–the smaller pieces, including words and phrases.
Take how we look at a photo as an entirety while also taking into notice the individual pieces involved. This is a very similar concept, which is not a coincidence at all. Identifying images and objects in methods very similar to human perception is a field in which neural networks are being trained to do. This is much more than a simple resemblance in identifying the configuration of a photo, and then that of a sentence.
In fact, this is not much about the language itself. The system has yet to understand the difference between the future continuous and future perfect in sentences, and it cannot break up words according to their etymologies. This is all about math and statistics, and lacking here is the human touch. It is admirable to simplify interpretation down to a task of mechanics, yet it remains a chilling matter. However, in this case one has to admit that only a mechanical translation is needed, and the issues of artifice and interpretation remains superfluous.
How to advance
The GNMT paper that describes the entire concept mentions several advances made in this regard, which of course are quite technical and maybe too difficult to understand. Simply, they reduce the required computational overhead for language processing through such a method and also avoiding the pitfalls involved in this path.
For instance, rare words are where the system usually chokes upon because the very nature of their rarity makes them too difficult to comprehend and making it more problematic to associate with others involved. GNMT actually resolves this dilemma by breaking words that are uncommon into smaller pieces that it considers as separate words, and from there begins to learn the necessary associations.
Limiting the math precision in this case and taking use of Google’s Tensor Processing Units makes the entire process easier by reducing actual computing time. Custom hardware blueprinted with neural network training as a vital necessity further helps in this regard.
While the systems of input and output are quite different, they continue to exchange data wherever they interface. This allows them enjoy co-training and thus forming an in-out process of a more unified nature. If you need more details, it is recommended to refer to the paper in the link above.
The end result is a system of high accuracy far more capable than phrase-based translators and nearing the level of human quality. One certain factor is that Google would not deploy anything on its public website without making sure of its quality, and in this case such a difficult task of translating from Chinese to English. Tests also showed very good results for French and Spanish, and rest assured that GNMT will expand its expertise in the needed direction very soon.
A negative feature is that no one truly understands how the system works, similar to the same difficulty found in many other predictive models rendered by machine learning.
GNMT is somewhat similar to other models of neural net, being a vast set of parameters that go through the process of training and are difficult to analyze.
Don’t be mistaken because some get the interpretation that there is no clue at all. The moving parts of phrase-based translators, large in numbers, are people-designed. Therefore, when a piece malfunctions or is out of date, it can simply be set aside. Knowing that neural networks design themselves essentially by using millions of iterations, if something does go wrong we can’t simply reach and replace a particular part. Training a system from scratch isn’t a small task, and yet it can be achieved at a quick pace. It is a high probability that this process will be achieved on a regular basis as upgrades are considered.
Google has invested heavily on machine learning, and this interpretation method–available on the web and available for mobile devices–is most probably the biggest demonstration to this date in the company’s public features. Despite the consensus over the complexity, mysterious nature and creepy feeling of neural networks, one can hardly argue against the effectivity they have shown so far.
Imagine if GNMT can one day translate in real-time. All those interpreters at the United Nations and everywhere else would be out of work. And rest assured we will witness such a transition in the not so distant future.