Understanding Where Machine Translation (MT) Makes Sense

One of the reasons that many find MT threatening I think, is the claim by some MT enthusiasts that it that it will do EXACTLY the same work that was previously done by multiple individuals in the “translate-edit-proof” chain without the humans, of course. To the best of my knowledge this is not possible today, even though one may produce an occasional sentence where this does indeed happen. If you want final output that is indistinguishable from competent human translation, then you are going to have to use the human “edit-proof” chain to make this happen.

Some in the industry have attempted to restate the potential of MT from Fully Automated High Quality Translation - FAHQT (Notice how that sounds suspiciously like f*&ked?) to Fully Automated Useful Translation – FAUT. However, in some highly technical domains it is actually possible to see that carefully customized MT systems can outperform exclusively human-based production, because it is simply not possible to find as many competent technical translators as are required to get the work done.

We have seen that both Google and Bing have gotten dramatically better since they switched from RbMT to statistical data-driven approaches, but the free MT solutions have yet to deliver real compelling quality outside of some Romance languages, and the quality is usually far from competent human translation. They also offer very little in terms of control, even if you are not concerned about the serious data privacy issues that their use brings to the user. It is usually worthwhile for professionals to work with specialists who can help them customize these systems to the specific purpose they are intended for. MT systems evolve and can get better with with small amounts of corrective feedback if they are designed from the outset to do this. Somebody who has built thousands of MT systems, across many language combinations, is likely to offer more value and skill than most can get from using tools like Moses building a handful of systems, or even the limited dictionary building input possible with many RbMT systems. And how much better can customized systems get than the free systems? Depending on the data volume and quality, it can range from small but very meaningful improvements to significantly better overall quality.

So where does MT make most sense? Given that there is a significant effort required to customize an MT system, it usually makes most sense when you have ongoing high volume, dynamically created source data and tolerant users or any combination thereof. It is also important to understand that the higher the quality requirements, the greater the need for human editing and proofing. The graphic below elaborates on this.

While MT is unlikely to replace human beings in any application where quality is really important, there are a growing number of cases that show that MT is suitable for:

• Highly repetitive content where productivity gains with MT can exceed what is possible with just using TM alone
• Content that would just not get translated otherwise
• Content that simply cannot afford human translation
• High value content that is changing every hour and every day but has a short shelf life
• Knowledge content that facilitates and enhances the global spread of critical knowledge, especially for health and social services
• Content that is created to enhance and accelerate communication with global customers who prefer a self-service model
• Real-time customer conversations in social networks and customer support scenarios
• Content that does not need to be perfect but just approximately understandable

So while there are some who would say that MT can be used anywhere and everywhere, I would suggest that a better fit for professional use is where you have ongoing volume, and dynamic but high value source content that can enhance international initiatives. To my mind, customized MT does not make sense for one-time, small localization projects where the customization efforts cannot be leveraged frequently. Free online MT might still prove of some value in these cases, to boost productivity, but as language service providers learn to better use and steer MT, I expect that we will see that they will provide translators access to “highly customized internal systems”  for project work, and the value to the translators will be very similar to the value provided by high quality translation memory.  Simply put – it can and will boost productivity even for things like user documentation and software interfaces.

It is worth understanding that while “good” MT systems can enhance translator productivity in traditional localization projects, they can also enable completely new kinds of translation projects that have larger volumes and much more dynamic content. While we can expect that these systems will continue to improve in quality, they are not likely to produce TEP equivalent output. I expect that these new applications will be a major source for work in the professional translation industry but will require production models that differ from traditional TEP production.

 

However, we are still at a point in time where there is not a lot of clarity on what post-editing, linguistic steering and MT engine refinement really involve. They do in fact involve many of the same things that are of value in standard localization processes , e.g. unknown word resolution, terminological consistency, DNT lists and style adjustments. They also increasingly include new kinds of linguistic steering designed to “train” the MT system to learn from historical error patterns and corrections. Unfortunately many of the prescriptions on post-editing principles available on LinkedIn and translator forums, are either linked to older generation MT systems (RbMT), systems that really cannot improve much beyond a very limited point or are linked to a specific MT system. In the age of data-driven systems new approaches are necessary and we have only just begun to define this. These new hybrid systems also allow translators and linguists to create linguistic and grammar rules around the pure data patterns. Hopefully we will see much better “user-friendly” post-editing environments that bring powerful error detection and correction utilities into much more linguistically logical and pro-active feedback loops. These tools can only emerge if more savvy translators are involved (and properly compensated) and we successfully liberate translators from dealing with the horrors of file and format conversions and other non-linguistic tedium that translation today requires. This shift to a mostly linguistic focus could also be much easier with better data interchange standards. The best examples of these are from Google and Microsoft rather than the translation industry, thus far.

Talking about standards, possibly the most worthwhile of the initiatives focusing on translation data interchange standards is meeting in Warsaw later this month. The XLIFF symposium IMO is the most concrete and most practical standards discussion going on at the moment and includes academics, LSPs, TAUS, tools vendors and large buyers sharing experiences. The future is all about data flowing in and out of translation processes and we all stand to benefit from real, robust standards that work for all constituencies.

The Continuing Saga & Evolution of Machine Translation

I recently attended the 7th IMTT Conference in Cordoba, Argentina. I especially enjoy the IMTT events because somehow they have found a content formula that works for both translators and LSPs. You get to see the translation supply chain communicate in real-time. The overall culture of their events is also usually very collaborative, and to my mind the place to see the most open and constructive dialogue between translators and agencies. Some may not be aware that Argentina has a particularly strong concentration of skilled humans who understand the mechanics of localization (especially in FIGS BrPt), and many of the agencies, even small ones, are able to work with pretty much every TM and TMS (Translation Management System) system in the market with more than a basic level of competence. Because of historical decisions made by @RenatoBeninatto many years ago and a great university educational system, Argentina has become a place with a comprehensive and sizable professional translation eco-system.

There were a few presentations that I found especially interesting, including a plenary presentation by Suzanne de Santamarina on the use of quality metrics. You can see some of the twitter trail here and here but basically Suzanne described her very active use of J2450 measurements to improve the dialogue on quality with customers and with her translators. While there clearly is effort and expense involved in implementing this as actively as she has, I think it dramatically improves the conversation regarding translation quality between all the participants, as it is very specific and impersonal and clear about what quality means.  It is also a means to build what she called “customer delight” which of course also includes a major service component.

Quality in a product or service is not what the supplier puts in. It is what the customer gets out (of the product/service) and is willing to pay for. A product is not quality because it is hard to make and costs a lot of money, as manufacturers typically believe. This is incompetence. Customers pay only for what is of use to them and gives them value. Nothing else constitutes quality…
~ Peter Drucker

Asia Online makes a software tool available to enable customers to calculate J2450 scores for this very reason.  It helps to move the discussion from inactionable complaints like “I don’t like the quality” or “the quality is not good”,  to practical error identification and resolution action steps like  “Is there a way to reduce frequency of the wrong terminology errors in the system?” Just as proper use of BLEU scores requires care and some expertise so does the use of J2450. Suzanne’s company’s regular and highly structured use of J2450 is such that they can really assess the linguistic quality from project to project with a precision that  few have. Her approach is refreshing in its clarity and precision and quite a contrast form the meandering inconclusive discussions on “quality” that you see in LinkedIn. Tools like BLEU and J2450 depend on the skill level of the user, and require an investment of time and effort and repeated use to develop real user competence before one understands the informational insights that their use can provide.

I also enjoyed the presentations by master translators like João Roque Dias and Danilo Nogueira  on the craft and art of translation, and enjoyed talking to them about MT and the life of the translator in general. (Yes, MT is sometimes useful even for some of them.) There were several skill focused presentations on Language QA tools, CAT and collaborative tools that were also very interesting. I heard great things about Val Ivonica’s presentation (in Brazilian) on translation productivity tools which I was unable to attend as it coincided with mine. It is interesting that Patricia Bown positioned MemoQ as collaboration software that enables the linear TEP model to evolve, enabling faster turnaround and higher volume. There were many Brazilians present (though some said not enough) and they lived up to their reputation for revelry but unfortunately were thwarted in their (our) attempts to find a karaoke place one evening. Nevertheless they shared their linguistically oriented humor with me and I had no difficulty finding a willing interpreter even though I was often the only person who did not speak the language.

I delivered a presentation on the emerging role of MT as a means to deal with the translation challenges created by the content explosion and new kinds of dynamic product/business related content. The feedback I received was mostly positive and constructive even though there were several very skeptical translators in the crowd. There were some in the audience who have already experienced MT that works and even those who had not worked with customized systems admitted that sometimes MT was useful.  I was also on a panel on “The Future of the Industry” which got mixed reviews as some translators felt it was not relevant and others felt it was a tired topic that nobody had any real clarity on. Many feel change is coming but are not clear what this really means and unfortunately for many the end-result of these changes is that customers expect more work for less money. This does mean that there is a certain amount of apprehension amongst the attendees as the future is not quite predictable.

A blog entry by Emily Stewart that pondered upon the theme of technology driven change at the conference a few days later, triggered an interesting and on-going discussion in LinkedIn.  Her post which was about the advent of technology in a variety of different markets is thoughtful and worth reading. I also think her conclusion (shown below) is good advice for us all.

Instead of denouncing machine translation as the end of the translation world as we know it, it may be time to take a step back and see what happens.  The discussion shouldn’t stop, but perhaps it could become less polemic and instead convert into a deeper conversation on and reflection of what may or may not lie ahead.

While initially there is a lot of focus on the perceived threat (there are some who think that I, together with other over zealous MT developers, am responsible for some of this fear and FUD), I am hoping that the dialogue moves beyond this point. Some MT systems have indeed been used to push rates down unfairly, but as we all begin to better understand these early mishaps, this can and must change. As George W Bush misspoke when he tried to say "Fool me once, shame on you; Fool me twice, shame on me."  (I hope you click on the Bush video, it is toooo funny). If it becomes clearer to everybody what it actually takes to “finish off” MT output to required target quality levels, this kind of abuse cannot continue. We need better quality assessment so that this gap can be more clearly defined.

All MT systems are not equal and to have a global post-editing pricing policy is guaranteed to create disenchantment. We all need to better understand where to use MT and where to avoid it. MT cannot easily if ever replace humans, on the same projects that were previously done through a careful human TEP process. If the quality expectations are high, it has to be MT and human.  MT makes most sense where there is ongoing volume and information volatility. We also need to better understand how to quickly assess the output quality of different MT systems so that post-editors are compensated fairly. The best MT systems are yet to come and they will be better because they are the product of informed linguistic steering in addition to standard data and MT techniques. We have yet to see useful compensation systems develop for these linguists and this will probably be needed before some of the uneasiness dissipates, but the forces driving this expanding need are strong and hopefully we should realize and recognize the value of these key individuals at some point in the future. This is already true at Asia Online so I imagine it can be done elsewhere.

In terms of disintermediation, I think MT will be only part of the whole picture, as we see more people learn to use motivated communities to get work done. Adobe and others have learnt to use “the crowd” to get traditional localization work done using translation platforms like Lingotek and newcomer Smartling (which might also have obtained the biggest startup investment made by a VC in the translation industry.) Much of the coming change will also come from collaboration software platforms like Lingotek, Smartling and others yet to come, that change how translation projects get done in terms of process flow, and that have a different modus operandi from traditional localization tools born in the TEP world. Translators are required to spend too much time working with data in different formats and too little time on the actual act of translation. New collaboration platforms and real data interchange standards will hopefully enable translators to focus mostly on real linguistic problem solving, and not on managing archaic and arcane format interchange issues. 

From my vantage point, I see that -
1) Translation is increasingly done outside of the sphere of the localization world and community based translation initiatives around the world are gathering momentum both in the non-profit and corporate world
2) The volume of translation that can help drive international business initiatives forward is increasing at a substantial rate (5X to perhaps as much as 100X) Interestingly, there are still some who think that this content explosion is a myth.
3) Social network conversations matter and are often more important to translate than having user documentation that is "perfect" and “error-free”. The company to customer communications have also become much more interactive, real-time and urgent and go way beyond the scope of most user documentation. 

Thus to approach every translation task with the TEP mindset that made great sense in the 1X or 2X volume days is not useful today. New approaches are needed and new models of automation/collaboration are necessary - and are perhaps the only way that all the changing momentum can be handled effectively. MT is simply one part of the equation and is far from being the whole solution. The need to solve this overall translation challenge is linked to the customers business survival so it gains a kind of momentum of inevitability. Businesses need to translate way more content to remain competitive in global marketplaces that move at internet speeds, thus automation and better collaboration are essential and critical to success and even survival.

 
We have seen in the last 5 years that many of the largest global organizations have launched MT initiatives on their own, because their LSP vendors were/are stuck in the TEP mindset, and did not realize that their customers had to learn to do dramatically more translation with not very much more money. This is perhaps a clue that in certain volume and time constraint scenarios, MT is necessary. We have seen that global enterprises need to solve this problem with or without vendors who historically managed the bulk of their localization translation. My sense is that this trend is likely to build momentum if LSPs do not learn to offer real MT competence. Real MT competence comes from building custom systems and seeing what works and what does not. Global enterprises will increasingly take this task upon themselves if they cannot find LSPs who can help them solve this problem e.g. TAUS is mostly a buyer driven organization with the key focus of sharing TM and facilitating large scale MT initiatives. The greatest successes presented at TAUS are all in-house initiatives with little LSP involvement. Surely this is because there is a real need, and we see that competitors are willing to share linguistic data and resources to handle this problem. I suspect that the buyer’s motivation is less about saving cents per word on translation costs, and much more about keeping and building customer loyalty and satisfaction in a world with growing global online commerce and information access needs.

My guess is that some of the anxiety on the coming change comes not so much from raw technology like MT, but perhaps it's real origin is the growing awareness that some of the work they are involved with grows less valuable to the customer’s real mission: which is to build and develop international markets. Perhaps the anxiety is really rooted in the fact that they sense that they are not involved in high value work. The real threat is not MT per se, but it is the growing awareness amongst international marketing executives (in global enterprises) that they need to focus on what their customers really care about - more and more often this is something other than getting a really great user manual out. Have you noticed that many leading edge companies like Apple, Sony have dramatically reduced their investment in user manuals? The iPhone simply does not have one (in the box but they do on the web). I am not suggesting that manuals are going away, but it is already clear that their relative value is diminishing. The content that drives global customer adoption and loyalty is changing and thus the relative value of traditional localization (software and documentation) work also changes.

I expect that new translation production models to build success in international markets will involve MT (and other translation automation), crowdsourcing as well as professional oversight and management. It is very likely that old production models like TEP will be increasingly less important, or just one of several approaches to translation projects as new collaboration models gain momentum.I think that the most successful approaches to solving these "new" translation problems  will involve a close and constructive collaboration between traditional localization professionals, linguists, MT developers, end-customers and probably others in global enterprise organizations who have never worked in "localization" but are more directly concerned about the quality of the relationship with the final customer across the world. At the end of the day our value as an industry is determined by how useful our input is to the process of building international markets and the requirements for success are changing as we speak.

The conversations at IMTT and the ensuing discussions suggest that while progress is being made in the understanding of translation technology, there is still a long way to go. I hope that at future IMTT conferences we see more discussion of approaches to translation projects where TEP may not make sense and automation and collaboration approaches can help solve different kinds of problems that also further international business initiatives. I expect that IMTT will be a leader in changing the current polemic and also expand the conversation to new stakeholders. This conversation is likely to require much more direct content with product management, international sales and support teams and the final end customer. Hopefully some of us in the industry get to lead or participate in  the driving this change through these new conversations.

While change can be difficult it can also be a time of opportunity and a time when leadership changes. Very few try to understand the forces of change better. People often go through a sequential emotional cycle before they learn to cope, and eventually even thrive when facing disruptive change. Those who get stuck at fear and despair, often end up as victims.

This little video shows that effective and heartfelt communication across cultures need not be heavily planned, ponderous or calculated. Sometimes simple and real is enough to create the change and build a connection to your customers.

Where the Hell is Matt? (2008) from Matthew Harding on Vimeo.

Translation Crowdsourcing

The phenomena of a crowd (or community) stepping forward and doing real translation work, often for no direct financial compensation is something that troubles many in the professional translation world. Mostly because they see this activity, as work being taken away from legitimate professionals or they see it as a ploy to reduce prices. While in some cases their fears may actually be justified, in the most successful uses of this approach I think it is clear that this is not true.

As I have said before, the growing momentum in the volume of content demands new production models. This momentum which exists both in the corporate world and also the general world out there simply cannot be addressed by ONLY using traditional professional translation production models. New needs require new approaches. For those who insist that the data deluge is a fiction, the rest of this post is probably irrelevant.

Another key driving force behind new crowdsourcing initiatives is the need to engage and interact with users in new markets. In new markets having active conversations with locals is key to building brand awareness and really learning about local needs and behavior. This is frequently a more important driver than cost containment as many in the industry think.

In some cases were it not for crowd-based localization efforts, it would simply not be done as it is not economically feasible to undertake the same efforts (and expenses) as are made for FIGS/CJK for “lesser” languages. Thus crowdsourcing is emerging sometimes as a means to get “lesser” languages done.

If we look at some of the most successful examples of crowd-sourced translation in practice, we can see that they have many if not all of the following elements in common.

A Crowd/Community That Is Invested:

·         TED Open Translation Project – Volunteer translators are often inspired by the content and wish to share it with their friends and countrymen. June Cohen has said that the volunteer translators in general do better quality work than the many of the paid professionals, who initially did a few translations to seed the project because of their passion for the subject and often their subject matter expertise. This effort has now enabled over 20,000 translations into 80+ languages of really challenging material. Many professionals also volunteer because they believe in the high general value of the content.

·         Facebook – Users who wish to build and expand the friend community in their particular language group. This effort has enabled Facebook to grow rapidly in international markets and accomplish very rapid coverage across 60+ languages. Had they used traditional means to do this it may have taken them years to get to the same point. Critics also often miss the point that engaging real users in the translation task also encourages rapid growth of the user base as “user translators” engage friends into their network.

·         Microsoft - MVPs (top accredited reseller partners) who wish to make technical support knowledge about Microsoft products more easily and widely available in their markets. Their efforts are rewarded by lower support costs and also an increase in product sales as more and more users look for self-service knowledge base information. Microsoft has been a trail blazer in making large amounts of knowledge base content available via MT, they are now adding crowd based editing to raise the quality of the translated information. Thus the most used and vital information tends to get the most attention and benefits all users.

·         Asia Online – Student users provide corrective feedback to continue to improve the translation quality of the Wikipedia and other knowledge content that is initially done by highly customized MT engines and paid translators. The students themselves will be the primary beneficiaries of this content, and their efforts will enable them to access high quality educational information. The volume of this information will likely increase a thousand fold.

 ·         Yeeyan:  has 150,000 registered users, who collectively translate 50 to 100 news articles every day from English to Chinese. Since its inception in 2006, the site has grown into a key gateway for Chinese speakers who want to follow international news. It has been so successful that it has attracted the attention of major news sources like The Guardian and ReadWriteWeb. Yeeyan is focused on addressing the problem of ghettoization of information by language through a community collaboration, where members both identify interesting content and also help to translate this content.

·          Adobe: This is a much more carefully managed effort designed to engage influential users, partners and customers to help provide relevant information for the broad Adobe User community in China.

T   Twitter: The translation center asks Twitter users -- all volunteers -- to help translate Twitter's interface into various languages. Once the basic support pages are translated, a select group of the "most active" translators are invited to work with Twitter to "maintain localized versions of the service." Twitter boasts that its translation center has 200,000 translators, and that the localization process for Dutch and Indonesian took just one month from the first call for involvement to its announcement. The availability of its interface in multiple foreign languages is bound to increase its popularity and effectiveness not only for online marketing but also for social and political activism.

Software Infrastructure That Facilitates Contribution & Participation:

In all of the cases above the companies involved crowdsourced translation initiatives need to invest in software that enables tasks to be parceled out, evolve as tasks change, enable efficient administration, maintain quality, gather feedback, and build self-sustaining eco-systems. The tools developed by dotSUB, Lingotek, Yeeyan and Asia Online are all unique collaboration and translation workflow management tools that enable these kinds of initiatives, They make little or no use of industry standard tools like Trados and TMS because of the highly proprietary, rigidity and archaic nature of these tools. These new-generation tools are much more open and are designed to evolve with technical and process advances on the internet today. It is quite possible that these community efforts could produce tools that supercede many of the tools in use today as these new tools focus on collaboration and sharing assets to enhance the efficiency of the collaborative translation process.

The Importance of Engagement and Higher Purpose:

It is interesting to note that translation is not the primary business of any of the companies listed in the examples above. In every case the goal and intent is to make more information available faster. Even for many of the corporations that are exploring crowdsourcing, the rationale is often more about customer engagement than cost savings. It is also important to note that none of these initiatives could even be attempted without the use of automation and large-scale community support and they are enabling initiatives that would not be possible otherwise. This is also true for Facebook who still had to use professionals to translate legalese that their community was not interested in translating.  The role of communities is likely to increase in future as more of the world comes online.

As we move forward we will see much more video and other rich content come online and already it is clear that the old approaches will not enable us to make this new content multilingual in an effective time frame. Crowdsourcing and automated translation will be necessary tools for an organization that seeks to communicate across the globe. As Clay Shirky has pointed out, the ‘cognitive surplus’ of the online population is a force that can be harnessed under the right circumstances and for the right purposes. It is likely that the professional translation world is going to see significant disruption in the coming years, as innovators figure out how to build sustainable models around community engagement, technology and organizational mission. However, as we have already seen, there is much that the crowd has no interest in doing and we should expect that this is not likely to change.

Crowdsourcing is here to stay and is a new mode of production that enables high–volume projects to be undertaken, engage with users and partners more deeply and participate in multilingual social networks where so many branding impressions are being formed. Managing crowdsourcing is also a major opportunity for savvy LSPs who have processes in place to recruit and manage the collaboration of dispersed volunteers and contributors.