The topic of post-editing MT (PEMT) (yes, somebody has already come up with an acronym) continues to gain prominence in the professional translation world and is also a subject of heated and sometimes disdainful discussion amongst many translators in the blogosphere. There is a lot of confusion and conflation in the discussion, and this post is an attempt to define and clarify the issues around this growing phenomenon, to see if it is possible to have a more constructive dialogue.
This is a subject of some importance, so I think it is worth exploring over a series of entries, but we can hopefully frame the key issues in this post. I should also say at the outset, that I maintain a perspective that assumes that much of the investment in MT is considered as creation of long-term translation production infrastructure rather than as something used for a single one-time project. This perspective I think makes MT at least somewhat different from most other “CAT Tools”.
There are at least three (and probably more) areas of general concern on PEMT:
- A clear definition of what PEMT actually is
- Compensation for PEMT work
- The quality of the work experience
What is the task of post-editing machine translation?
The most commonly understood definition of PEMT in the localization world is described graphically below. It is usually understood to be the linguistic work needed to correct MT output to a linguistic quality level that is close to, or indiscernible from a standard TEP process. At it’s worst, this corrective work can be very tedious and perhaps even a mind-numbing process.
However, this view neglects several factors that affect the actual PEMT process, e.g.
- MT system output varies, and some MT systems produce much better output than others, and thus the PEMT experience can be very different and is highly dependent on the actual quality produced by each individual MT system or engine. “Good” MT engines (systems) can noticeably enhance the productivity of translators and vice versa.
- Google Translate is often not the example of the best that MT can do, and customized, domain focused system usually outperform generic engines.
- Increasingly, MT is used to translate content that would never be translated through a standard TEP process. Usually this involves much larger volumes of content that is also frequently updated and changed.
- There are many MT applications where the editing work is only focused on making sure the content is understandable and accurate in meaning, even if it is grammatically imperfect.
- It is possible to do linguistic analysis and a priori terminology and linguistic work to enhance the ability of the MT system to produce better output, and this can also be considered a kind of PEMT activity.
- Many equate PEMT to “janitorial” work (often accurately so) that no self respecting translator would ever resort to, but there are many different bilingual skill levels required in the development process of an MT engine and very few of the critics seem to realize this.
Thus, I would expand the definition of PEMT to:
All tasks that are intended to improve the linguistic quality of output produced by an MT engine. This includes both the a priori analysis and post-editing and structural linguistic analysis work involved in the development of an MT engine. The objective always being to improve translator productivity (as well as reducing cost and time) on every translation project in that domain in future.
The experience at Asia Online is more closely characterized by the following graphic which shows a rapidly (weeks) evolving MT system that produces continuously improving MT quality as corrective feedback is fed back into the MT engine and error patterns are identified and gradually eliminated. This is a process that has been in use in the Asia Online project to translate the English Wikipedia to Thai. The processes are transferable to other languages. There is a highly collaborative process underlying the MT systems here where linguists are looking to eliminate as many linguistic error patterns as possible and thus also enhance the ongoing PEMT experience and process, and expedite the translation of a billion word corpus and bring it up to human quality levels in the quickest time possible.
It is also often assumed that translators are the only people who are capable of doing post-editing work. However, many translators do not care for the “janitorial” aspect of the work so it is not for everybody. MT systems that produce generally “high-quality” output could also be edited by monolingual speakers of the target language with expertise in the subject domain. Thus, it is possible to use humans who are less skilled than your average translator to accomplish business objectives, especially where there are really large volumes involved and/or when grammatical perfection matters less than immediate access. There are many students and housewives that may actually find the flexibility and money offered by PEMT tasks attractive, and for very large ongoing projects they may indeed have a role. Problems arise when it is assumed that the work done by professionals can easily be done by non-professionals, so it is wise to be clear about your objectives and understand the nature of the work being offered and the skills required for competent performance. In general, open-minded professionals will always have an edge in producing better quality, but bringing in hostile or reluctant translators is also a sure way to fail.
The issues that get the most attention in the broader discussions on PEMT are the other two issues which are stated in brief below. These two issues will be examined in more detail in future posts.
Compensation for PEMT: The early experience in the industry has been to arbitrarily reduce standard rates for PEMT work as it is assumed that to some extent the translation is already done. This practice causes a lot of resistance amongst translators who are expected to actually produce a usable translation (at low rates) when the starting point MT output is not useful or viable. It is already true that the unfortunate state of affairs today is to link payment of per word rates to TM matching rates. This has caused commoditization of the actual translation work, and translators today are expected to track baskets of words and get paid different rates in schemes such as the hypothetical one shown below:
| Matching Rate | Pay Rate Per Word |
| Repetitions | 25% |
| 100% | 25% |
| 85% to 99% | 45% |
| 75% to 84% | 60% |
| 0 to 74% | 100% |
MT is sometimes used as a way to push these rates even lower. Thus, to be fair to translators there needs to be an assessment of the scope of the PEMT task. An MT system that produces 50% usable output should be compensated differently from one the produces 75% usable output, assuming the content has to be raised to the same target quality level. Logically, the greater the quality gap that needs to be filled, the greater the pay rate for doing the work. To do this accurately and fairly, we require rapid and widely accepted MT engine quality measures which do not exist today. As our understanding of this issue evolves, I would not be surprised to see more hourly, consulting fee and project based payment schemes develop in future, as linguists and tech-savvy translators get more involved in steering MT engine development initiatives.
The Nature Of PEMT Work: Another common complaint about PEMT is about the drudgery and mind-numbing nature of error correction work. (This seems surprising to me ;-) as I notice that many translators love to correct and point out linguistic, typo and errors of expression errors in blog comments and social networking discussions.) But this does however suggest that not all translators want to do this kind of work or are well suited to it. I have seen very positive and very negative translator feedback, but often the implementation of this technology has as much to do with it, as the work process itself. Success with this technology often requires close collaboration with key translators who often provide great value in the process. And of course they must be compensated for these contributions. For very large projects it will become necessary to use less skilled workers or the “crowd”. I think that the tools will improve and possibly even make translation work more fun, whole and organic. (The fragmentation caused by the current TM matching rate approach has got to be a nightmare for many.) TAUS offers some guidelines for PEMT and this is a subject of great and growing interest to many. There are many levels of human steering interaction possible with MT and as some translators engage more often with MT systems they will start to understand where they have a long-term role to play as quality drivers. I expect that they will become critical members of teams who undertake to make massive content repositories increasingly multilingual.
There are in fact environments where MT systems have been developed in close collaboration with translators, and thus offer clearly understood productivity advantages. In these cases the translators really want to use the systems and would see not having access to the MT system as a disadvantage e.g. PAHO, Asia Online, Andovar. Most of the time, MT has yet to really rise to a level where it is a must-have tool for a professional translator but I think that is the direction we are heading in. The content deluge shows no sign of slowing down, and any global enterprise worth anything realizes, that they need to translate a whole lot more than user manuals and spec sheets if they want to build strong international businesses in future. So this is worth some attention and worth doing well.
