The Evolving Translator-Computer Interface

This is a guest post by Nico Herbig from the German Research Center for Artificial Intelligence (DFKI).

For as long as I have been involved with the translation industry, I have wondered why the prevailing translator machine interface was so arcane and primitive. It seems that the basic user interface used for managing translation memory was borrowed from DOS spreadsheets and has eventually evolved to become Windows spreadsheets. Apart from problems related to inaccurate matching, the basic interaction model has also been quite limited. Data enters the translation environment through some form of file or text import and is then processed in a columnar word processing style. I think to a great extent these limitations were due to the insistence on maintaining a desktop computing model for the translation task. While this does allow some power users to become productive keystroke experts it also presents a demanding learning curve to new translators.

Cloud-based translation environments can offer much more versatile and powerful interaction modes, and I saw evidence of this at the recent AMTA 2020 conference (a great conference by the way that deserves much better social media coverage than it has received.) Nico Herbig from the German Research Center for Artificial Intelligence (DFKI) presented a multi-modal translator environment that I felt shows great promise in updating the translator-machine interaction experience in the modern era. 

 

Of course, it includes the ability to interact with the content via speech, handwriting, touch, eye-tracking, and seamless interaction with supportive tools like dictionaries, concordance databases, and MT among other possibilities. Nico's presentation focuses on the interface needs of the PEMT task, but the environment could be reconfigured for scenarios where MT is not involved and only used if it adds value to the translation task. I recommend that interested readers take a quick look through the video presentation to get a better sense of this.

*** ======== ***

MMPE: A Multi-Modal Interface for Post-Editing Machine Translation

As machine translation has been making substantial improvements in recent years, more and more professional translators are integrating this technology into their translation workflows. The process of using a pre-translated text as a basis and improving it to create the final translation is called post-editing (PE). While PE can save time and reduce errors, it also affects the design of translation interfaces: the task changes from mainly generating text to correcting errors within otherwise helpful translation proposals, thereby requiring significantly less keyboard input, which in turn offers potential for interaction modalities other than mouse and keyboard. To explore which PE tasks might be well supported by which interaction modalities, we conducted a so-called elicitation study, where participants can freely propose interactions without focusing on technical limitations. The results showed that professional translators envision PE interfaces relying on touch, pen, and speech input combined with mouse and keyboard as particularly useful. We thus developed and evaluated MMPE, a CAT environment combining these input possibilities. 

Hardware and Software

MMPE was developed using web technologies and works within a browser. For handwriting support, one should ideally use a touch screen with a digital pen, where larger displays and the option to tilt the screen or lay it on the desk facilitate ergonomic handwriting. Nevertheless, any tablet device also works. To improve automatic speech recognition accuracy, we recommend using an external microphone, e.g., a headset. Mouse and keyboard are naturally supported as well. For exploring our newly developed eye-tracking features (see below), an eye tracker needs to be attached. Depending on the features to explore, a subset of this hardware is sufficient; there is no need to have the full setup. Since our focus is on exploring new interaction modalities, MMPE’s contribution lies on the front-end. At the same time, the backend is rather minimal, supporting only storing and loading of files or forwarding the microphone stream to speech recognition services. Naturally, we plan on extending this functionality in the future, i.e., adding project and user management functionality, integrating Machine Translation (instead of loading it from file), Translation Memory, Quality Estimation, and other tools directly in the prototype.

Interface Layout

As a layout, we implemented a horizontal source-target layout and tried to avoid overloading the interface. On the far right, support tools are offered, e.g., a bilingual concordancer (Linguee). The top of the interface shows a toolbar where users can save, load, and navigate between projects, and enable or disable spell checking, whitespace visualization, speech recognition and eye-tracking. The current segment is enlarged, thereby offering space for handwritten input and allowing users to view the context while still seeing the current segment in a comfortable manner. The view for the current segment is further divided into the source segment (left) and tabbed editing planes for the target (right), one for handwriting and drawing gestures, and one for touch deletion & reordering, as well as a standard mouse and keyboard input. By clicking on the tabs at the top, the user can quickly switch between the two modes. As the prototype focuses on PE, the target views initially show the MT proposal to be edited. Undo and redo functionality and segment confirmation are also implemented through hotkeys, buttons, or speech commands. Currently, we are adding further customization possibilities, e.g., to adapt the font size or to switch between displaying source and target side by side or one above the other.

Handwriting

Hand-writing in the hand-writing tab is recognized using the MyScript Interactive Ink SDK, which worked well in our study. The input field further offers drawing gestures like strike-through or scribble for deletions, breaking a word into two (draw a line from top to bottom), and joining words (draw a line from bottom to top). If there is a lack of space to hand-write the intended text, the user can create such space by breaking the line (draw a long line from top to bottom). The editor further shows the recognized input immediately at the top of the drawing view. Apart from using the pen, the user can use his/her finger or the mouse for hand-writing, all of which have been used in our study, even though the pen was clearly preferred. Our participants highly valued deletion by strike-through or scribbling through the text, as this would nicely resemble standard copy-editing. However, hand-writing for replacements and insertions was considered to work well only for short modifications. For more extended changes, participants argued that one should instead fall back to typing or speech commands.

Touch Reorder

Reordering using (pen or finger) touch is supported with a simple drag and drop procedure: Users have two options: (1) They can drag and drop single words by starting a drag directly on top of a word, or (2) they can double-tap to start a selection process, define which part of the sentence should be selected (e.g., multiple words or a part of a word), and then move it. 

We visualize the picked-up word(s) below the touch position and show the calculated current drop position through a small arrow element. Spaces between words and punctuation marks are automatically fixed, i.e., double spaces at the pickup position are removed, and missing spaces at the drop position are inserted. In our study, touch reordering was highlighted as particularly useful or even “perfect” and received the highest subjective scores and lowest time required for reordering. 

 

Speech

To minimize lag during speech recognition, we use a streaming approach, sending the recorded audio to IBM Watson servers to receive a transcription, which is then interpreted in a command-based fashion. The transcription itself is shown at the top of the default editing tab next to a microphone symbol. As commands, post-editors can “insert,” “delete,” “replace,” and “reorder” words or sub-phrases. To specify the position if it is ambiguous, anchors can be specified, e.g., “after”/”before”/”between” or the occurrence of the token (“first”/”second”/”last”) can be defined. A full example is “replace A between B and C by D,” where A, B, C, and D can be words or sub-phrases. Again, spaces between words and punctuation marks are automatically fixed. In our study, speech [recognition] received good ratings for insertions and replacements but worse ratings for reorderings and deletions. According to the participants, speech would become especially compelling for longer insertions and would be preferable when commands remain simple. For invalid commands, we display why they are invalid below the transcription (e.g., “Cannot delete the comma after nevertheless, as nevertheless does not exist”). Furthermore, the interface temporarily highlights insertions and replacements in green, deletions in red (the space at the position), and combinations of green and red for reorderings. The color fades away after the command. 

Multi-Modal Combinations of Pen/Touch/Mouse&Keyboard with Speech

Multi-modal combinations are also supported: Target word(s)/position(s) must first be specified by performing a text selection using the pen, finger touch, or the mouse/keyboard. 

Afterwards, the user can use a voice command like “delete” (see the figure below), “insert A,” “move after/before A/between A and B,” or “replace with A” without needing to specify the position/word, thereby making the commands less complex. In our study, multi-modal interaction received good ratings for insertions and replacements, but worse ratings for reorderings and deletions. 

Eye Tracking

While not tested in a study yet, we currently explore other approaches to enhance PE through multi-modal interaction, e.g., through the integration of an eye tracker. The idea is to simply fixate the word to be replaced/deleted/reordered or the gap used for insertion, and state the simplified speech command (e.g., “replace with A”/”delete”), instead of having to manually place the cursor through touch/pen/mouse/keyboard. To provide feedback to the user, we show his/her fixations in the interface and highlight text changes, as discussed above. Apart from possibly speeding up multi-modal interaction, this approach would also solve the issue reported by several participants in our study that one would have to “do two things at once” while keeping the advantage of having simple commands in comparison to the speech-only approach.

Logging

MMPE supports extensive logging functionality, where we log all text manipulations on a higher level to simplify text editing analysis. Specifically, we log whether the manipulation was an insertion, deletion, replacement, or reordering, with the manipulated tokens, their positions, and the whole segment text. Furthermore, all log entries contain the modality of the interaction, e.g., speech or pen, thereby allowing the analysis of which modality was used for which editing operation. 

Evaluation

Our study with professional translators showed a high level of interest and enthusiasm about using these new modalities. For deletions and reorderings, pen and touch both received high subjective ratings, with the pen being even better than the mouse & keyboard. Participants especially highlighted that pen and touch deletion or reordering “nicely resemble a standard correction task.” For insertions and replacements, speech and multi-modal interaction of select & speech were seen as suitable interaction modes; however, mouse & keyboard were still favored and faster. Here, participants preferred the speech-only approach when commands are simple but stated that the multi-modal approach becomes relevant when the sentences' ambiguities make speech-only commands too complex. However, since the study participants stated that mouse and keyboard only work well due to years of experience and muscle memory, we are optimistic that these new modalities can yield real benefit within future CAT tools.

Conclusion

Due to continuously improving MT systems, PE is becoming more and more relevant in modern-day translation. The interfaces used by translators still heavily focus on translation from scratch, and in particular on mouse and keyboard input modalities. Since PE requires less production of text but instead requires more error corrections, we implemented and evaluated the MMPE CAT environment that explores the use of speech commands, handwriting input, touch reordering, and multi-modal combinations for PE of MT. 

In the next steps, we want to run a study that specifically explores the newly developed combination of eye and speech input for PE. Apart from that, longer-term studies exploring how the modality usage changes over time, whether translators continuously switch modalities or stick to specific ones for specific tasks are planned. 

Instead of replacing the human translator with artificial intelligence (AI), MMPE investigates approaches to better support the human-AI collaboration in the translation domain by providing a multi-modal interface for correcting machine-translation output. We are currently working on proper code documentation and plan to open-source release the prototype within the next months. MMPE was developed in a tight collaboration between the German Research Center for Artificial Intelligence (DFKI) and Saarland University and is funded in part by the German Research Foundation (DFG).

Contact

Nico Herbig - nico.herbig@dfki.de

German Research Center for Artificial Intelligence (DFKI)

Further information:

Website: https://mmpe.dfki.de/

Paper and additional information:

Multi-Modal Approaches for Post-Editing Machine Translation
Nico Herbig, Santanu Pal, Josef van Genabith, Antonio Krüger Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM 2019
ACM Digital Library - Paper access

(Presenting an elicitation study that guided the design of MMPE)

MMPE: A Multi-Modal Interface using Handwriting, Touch Reordering, and Speech Commands for Post-Editing Machine Translation
Nico Herbig, Santanu Pal, Tim Düwel, Kalliopi Meladaki, Mahsa Monshizadeh, Vladislav Hnatovskiy, Antonio Krüger, Josef van Genabith Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. ACL 2020
ACL Anthology - Paper access

(Demo paper presenting the original prototype in detail)

MMPE: A Multi-Modal Interface for Post-Editing Machine Translation
Nico Herbig, Tim Düwel, Santanu Pal, Kalliopi Meladaki, Mahsa Monshizadeh, Antonio Krüger, Josef van Genabith Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. ACL 2020
ACL Anthology - Paper access - Video

(Briefly presenting MMPE prototype and focusing on its evaluation)

Improving the Multi-Modal Post-Editing (MMPE) CAT Environment based on Professional Translators’ Feedback
Nico Herbig, Santanu Pal, Tim Düwel, Raksha Shenoy, Antonio Krüger, Josef van Genabith Proceedings of the 1st Workshop on Post-Editing in Modern-Day Translation at AMTA 2020. ACL 2020
Paper access - Video of presentation

(Recent improvements and extensions to the prototype)

3 Ways You Can Become an ‘Augmented Translator’

This is a guest post by Jonathan Grisot who works as a Senior Language Specialist in SDL’s French Language Office in Paris.

The translators working in the Language Offices at SDL are probably amongst the most tech-savvy pool of translators in the business translation industry. Their proximity to developers of a range of translation technology, ranging from TM (Translation Memory) to TMS (Translation Management Systems), to MT (Machine Translation) and other translation process-related tools gives SDL internal translators a natural advantage. The possibilities and potential for transforming business translation processes and outcomes have gone into high gear in the last four years, as a new management team began an overhaul and rationalization initiative that has already been yielding results. In the coming years, we should expect to see much new innovation coming from these strategic efforts. 

SDL is unique amongst the largest 50 LSPs in having the largest pool of in-house translators.  After the initial wave of euphoria around machine learning and "AI," the world is beginning to realize that the best results are often better accomplished by new models of (wo)man-machine collaboration than by machine alone. As tool developers and linguists begin to engage more intensively and as the dialogue shifts from FUD related factors, to true problem-solving focused collaboration, a new breed of technology and services is emerging.  

I have always felt that the best MT technology will evolve from a productive dialogue between competent translators and open-minded MT technology experts. Superior solutions emerge from process innovation entwined with relevant NLP capabilities and expertise, for the purpose of solving high-value business problems. Long-term experience in solving these problems can provide the insight needed to find optimal process-technology-expertise combinations to address the issues at hand. There are use cases where only an expert human would work and there are use cases where machines alone can handle the sheer scale, volume, and timeliness that is needed. However, in this age of digital transformation where the modern enterprise has content as the core of their digital footprint, the role of properly deployed MT can only grow. This will require a man-machine collaboration that adjusts and adapts to the varying needs and shades of grey that are needed to solve these new kinds of translation challenges.    

This post is a perspective from the translator frontline at SDL and represents an attitude that will produce superior services and technology. SDL has a broad and deep engagement with global enterprises around content management and translation challenges and it is clear that the approach needed for the future will by necessity need to be more varied. The reality before us is nicely captured by research shared in the 2019 United Nations Conference on Trade and Development Digital Economy Report, which shows that global internet protocol (IP) traffic, a proxy for data flows, grew from about 100 gigabytes (GB) per day in 1992 to more than 45,000 GB per second in 2017. By 2022, the figure is expected to stand at 150,700 GB per second.

======

The future of professional translation is here. Are you ready? Translation is driving the globalization of communication, but it encompasses more than just translation: linguistic advising, review, proofreading, transcreation, subtitling, language consultancy, linguistic content management... the list goes on. No doubt 2020 and beyond is set to increase opportunities for translators to add value to their clients. But, given the rapidly changing world we now live in, how can translators evolve their own services, becoming 'Augmented Translators'?

Engage with technology

The purpose of technology in translation has always been to help translators deliver and finalize content faster. The days when translators were locked up in a library with a pile of dictionaries and a pencil to produce a translation are long gone. Today, content is processed online, from brochures and web pages to user manuals and market outlooks. Even traditional white papers are no longer exclusively published in paper format. And the list of tools, plug-ins, and technologies available to help translators to finalize and reach audiences continues to grow: translation memories, terminology databases, fragment matches, upLIFT, Neural Machine translation, Autosuggest dictionaries, and more.

Even our corporate language is changing with technology: instead of “engaging” with customers, companies “connect” with customers. Now, for those who are familiar with the technologies offered by our flagship solution, SDL Trados Studio, check out the many assumptions raised about our future by language specialists here.

Even more tech-savvy? Check the other side of the fence and see how content will impact the augmented translators’ environment. Discover SDL Content Assistant, a technology that was considered science-fiction several years ago – but is now very real.

Also, with today’s technology, the help provided to translators does not only come from the tools: now, even content creates itself. Now, it’s up to us, translators, to transform it for our local audience.

Specialize in quality levels, not only in specific industries

Fact: the amount of content to translate has reached incredible levels. While SDL translates hundreds of billions of words every year, this figure remains a drop in an ocean of all translated words. What matters is not the amount to translate. What matters is that the result displayed to your audience meets the quality level expected for such content.

However, billions of words also mean billions of possibilities, and augmented translators are aware of one truth that is the current state of affairs: there is no “standard translation”.

All translations are unique, because clients have unique needs, like their customers. And they also have unique constraints, terminologies, processes, and practices.

With the client’s needs in mind, the augmented translator will adjust their effort and the amount of time required to complete their tasks. And the productivity tools available nowadays are here to help them alleviate the burden: the augmented translator never translates from scratch.

The key factor here is to find the perfect dosage in productivity, the right balance between effort and result. It is important to have a strong understanding of the translation workflow, the tools and assets at your disposal, and your own strengths and skills. This will help assess the quality and thus reduce risks.

In fact, “quality” can only be assessed by a human mind, and this is where the augmented translator and the client can collaborate to set expectations on quality. Because both clients and translators know that a “lack of quality” also means “rework”. And while a “high-quality translation” may be expensive, a “low-quality translation” may cost even more.

Inject culture, and acquire knowledge

Augmented translators will speed up the process of integrating their clients’ requirements to get the quality needed, and that is a truth for all industries. But only if they have adequate assets to help them get started in an augmented world.

An augmented translator will take advantage of the following resources:

• Content reuse from translation memories
• Glossaries to apply preferred terminology
• Style guides to comply with formatting, grammar, and stylistic rules 
• The tone of voice or brand guides to convey the brand’s message 
• Project-specific instructions, like character limitations
• Machine and AI-enabled translation engines to accelerate productivity

All these automated tools and assets are literally “knowledge providers” to the translator, and help non-specialized translators to meet client requests even without even knowing the client. These knowledge providers are useful since all the clients have preferred terms, favorite wordings, and different rules.

Of course, this automation can also be error-prone and full of traps: terms in glossaries that do not take the context into account, incorrect source texts written by non-native speakers, corporate jargon not understandable outside of your client’s professional sphere, and more.

This is where the augmented translator has two strong cards to play: culture and understanding.

Augmented translators will be able to spot errors in the source text, avoid using offensive or restrictive content, use the appropriate language for the target audience, rewrite puns, detect dual meanings, adapt to stylistic rules, and correct erroneous terminology used by translation engines, etc.

The augmented translator walks in the footsteps of the ancient copyists and scribes and embraces the same mission and ambition: connect cultures and content to share a message.

*****

Jonathan Grisot started as a translator in 2007 and currently holds a position of Senior Language Specialist for SDL in Paris. He is responsible for driving Machine Translation initiatives, managing internal training and quality best practices and is still involved in various translation and transcreation projects. He is also managing the Junior Academy, a local SDL onboarding structure for newly hired SDL translators. Born in Burgundy and raised on the French Riviera, Jonathan considers his detective novels, sci-fi and fantasy books as his numerous children.

Sharing Efforts to get the most from MT and Post-Editing

This is a guest post from Luigi Muzii which is basically made up primarily of the speaker notes of his presentation at the ELIA Together 2018 conference. I believe that Luigi has wise words and represent best practices thinking and thus offer it on this blog forum.

While some of his advice might seem obvious to some, I am always struck by often these very commonsensical recommendations are either overlooked or simply ignored willfully.

As generic MT improves, I think it becomes more and more important for enterprises to consider bringing rampant, uncontrolled MT use by employees and outsourced translators under control. 

As always the emphasis of bold text in the post is my doing.

====

The presentation was designed to provide some practical advice about tackling the challenges that freelancers, project managers, and translation buyers face when approaching MT, implementing MT or running MT and post-editing projects.

Sharing efforts to get the most from MT+PE from Luigi Muzii

Three target groups can be identified for three kinds of task:

1. In the making; machine translation is used for “suggestions” while processing a document by a human translator; this is probably the most common approach today;
2. Downstream; machine translation is used to fully process a document that will be possibly post-edited; this approach is typically adopted by larger clients with established experience in the field;
3. On constraints; machine translation is used by an LSP to finalize a translation job by asking translators to work on suggestions coming from a specialized engine.

While scenarios two and three might meet the customer’s need for confidentiality and IP protection through an in-house engine using only the client’s own data, scenario one is becoming more and more general among professional translators, given the astonishing improvement of online engines. At the same time, scenario three is slowly but constantly applying to LSPs who try to escape price and volume pressures through machine translation and post-editing.

Laying foundations

The three scenarios just outlined require different strategies to be devised. The first one involves the machine-translation method.

Given the circumstances, the premises and the many reservations about it—basically, the hype—a question must be asked: Is PEMT already in the past?

In this paper, the reference method is PB-SMT (Phrase-Based Statistical Machine Translation) because PB-SMT engines are inexpensive and effective, whereas customizable NMT (Neural Machine Translation) engines are still quite pricey and challenging as to technical requirements and operational complexity, thus out of range for most customers. Translators working on unrestricted documents (scenario one applies here,) would generally choose an online NMT engine. For customers requiring confidentiality and IP protection and willing to leverage their own language data (scenario two and three apply here,) a highly customizable PB-SMT engine might be a valid option, especially where no major investment is envisaged in the field, vast and suitable data is available and/or limited volumes are processed.

In general, the main drivers in the adoption of MT are productivity (speed and volumes) and usefulness (consistency and marginal cost) especially for large projects otherwise involving many translators. Unfortunately, MT is not exactly child’s play. MT engines are complex and challenging applications requiring a combination of tools, data, and knowledge. This is a rare commodity, especially in a single person, on both sides of a translation project.

Also, in the future, while MT will continue to proliferate, the shortage of good translators will increase.

Join forces

Therefore, joining forces is important to explore and vet as many solutions as possible. According to a popular saying, everyone’s needed, no one’s indispensable, and can be easily replaced by anyone else with a similar profile in the same role. This also means, though, that, to be valuable, everyone’s effort is needed, on the highest level of performance all the time. For quite some time now, translation is no longer a solitary feat, but a collaborative effort. This is especially true with the current level of technology.

In this respect, three steps should be completed before starting any MT project.

1. Recap your goals and requirements
   What you expect from MT and how much you are willing to rely on it.
2. Check your MT readiness;
   Realistically analyze your knowledge, tools, and data.
3. Plan for assistance
   Never venture into an unfamiliar territory without a guide.

Defining requirements

When planning to implement MT, keep scope, goals, and expectations clearly distinct.
Identify one or more items within your scope of business for MT, possibly picking those where the amount of data available is larger, and the quality higher.

Clearly define and prioritize your goals. Major goals may be reducing labor, boosting productivity and keeping consistency, especially in larger projects.

Be realistic in your expectations. Therefore, familiarize with technology and strengthen your expertise to make the best of it. Tackle any security issues for confidentiality and data integrity and protection of intellectual property. Don’t forget to scrub your data if you plan to train an engine and to plan for any relevant support. Finally, revise your pricing model to encompass MT-related tasks.

Building a platform

When building an MT platform, never forget that MT engines are not all equal, for different environments, configurations, and data. Therefore, although the output could be considered someway predictable, raw output quality can vary across systems and language combinations, and error may not follow a consistent pattern. Performances of MT engines also vary.

In data-driven MT, data maintenance is crucial, and it is the first task when setting up an MT platform. Data must be organized, cleaned, and fixed for terminology and style. For a customized engine, at least 100,000 paired segments are necessary and the cleaner and healthier the better.

Another important factor to the effectiveness of an MT strategy are the tools used for data preparation, pre-translation, and post-editing. Special attention must be given to translation tool settings to allow for sub-segment recall and fuzzy match repair.

Finally, when choosing the engine, the items to consider are:

• The total cost of ownership (TCO)
• Integration
• Expertise
• Security (especially as to intellectual property and confidentiality)

 

Best practices: Running projects

When running MT projects, best practices may be different depending on whether you are a translation buyer or vendor.

In general, knowing your data and mastering quality metrics is a must. As to post-editing, devise your own compensation scheme.

A common mistake is to consider all content as equal and then mess with data. In the same way, absolutely avoid relying only on your capacities or on vendors. In the end, everything can be summed up in the simple invitation to not expect any miracles.

Never forget to agree with the customer and the content owner about using MT, especially if you are using a SaaS/online platform to prevent being sued.

Also, remember that data is the fuel to any SMT/NMT engine and that the output is only as good as the data used. In fact, these engines perform statistical predictions by inference, and when the amount and quality of data increase, an engine improves.

Collect as much data as possible, but always make sure it comes from few reliable sources in a restricted domain, that it contains correct translations with no errors, it is real and recent, and terminologically and stylistically consistent.

At this point, you must accept that MT output is unpredictable. For this reason, MT quality assessment should be run in such a way as to prevent any subjectivity.

For the same reason, post-editing is and will remain a critical, integrated part of MT usage, and it is expected to be fast, unchallenging, and flowing.

Anyway, the amount of post-editing required can be hard to assess. To plan deadlines and allocate a budget for the job, two different measures can be used, the edit time and the post-editing effort. The first is the time required to get a raw MT output to the desired standard, and the latter is the percentage of edits to be applied to raw MT output to attain the desired standard.

The main problem with edit time is that it can only be computed downstream, assuming that the time spent has been entirely devoted to editing.

The post-editing effort can be estimated through probabilistic forecasts based on automatic metrics as a reverse projection of the productivity boost. In fact, translation productivity is measured as the throughput expressed in the number of words per hour, and MT is supposed to improve it by reducing the turnaround time and increasing the workable volumes. However, the post-editing effort and the turnaround time are hard to predict, especially for translation of publishable quality and/or data for incremental training of engines. In fact, it depends on diverse factors such as the quality expectations for the finalized output, the volume of content to process, and the allotted time for the task. Also, the effort required depends on the post-editing level.

The post-editing level is generally restricted to:

1. Gisting;
2. Light;
3. Full.

Gisting consists in fixing recurring errors in raw MT output with automatic scripts. It is used for volatile content, e.g. messaging, conversations, etc. Light post-editing consists of fixing capitalization and punctuation errors, replacing unknown words, removing redundant words or inserting missing ones, and ignoring all stylistic issues. It is generally used for continuous delivery and reprocessing. Full post-editing consists in fixing meaning distortion, grammar, and syntax, translating untranslated terms (possibly new terms), and adjusting fluency. It is reserved for publishing and engine training.
Finally, always follow a few basic rules before boarding on a post-editing project:

• Test before operating;
• Ask for MT samples for negotiation;
• Negotiate throughput rates;
• Ask for glossary with the list of DNT words;
• Ask for instructions;
• Be open to feedback.

Similarly,

• Never use MT to sustain price competition;
• Never process poor MT outputs;
• Never treat post-editing as fuzzy matches.

Remember that different engines, domains, and language pairs produce different outputs, involve different post-editing efforts, and require different post-editing instructions. These should address tool selection criteria and environment setup guidelines, as well as a style guide, and a comprehensive term base. They should also address conventions and the type and number of project details as well as the general pricing model and the actual operating instructions.

As to pricing and compensation, for light post-editing of very good output when speed is a major concern and the first requirement, a model should be settled prior to any assessment of the actual MT output based on a clear-cut predictive scheme. However, do not follow any translation-memory fuzzy matches scheme. In fact, while fuzzy matches over 85% are inherently correct and generally require minor changes, machine-translated segments may contain errors and inaccuracies, and even a supposedly light post-editing may prove challenging. A downstream computation scheme might also be devised in full post-editing for an accurate measurement of the actual work performed. This is usually made by computing the edit distance and then inferring the percentage on the hourly rate.

A negotiation grid can be helpful to cross-reference type and nature of engines, quality of raw output, and all the relevant technical requirements with compensation based on productivity, type of performance, bid (hourly rate) and ancillary services (e.g. filling in QA forms for ongoing training of engine.)

In any case, a strong and clear “No, thanks!” is more than reasonable when a considerably low pay rate is offered that is unrelated to language pair and MT output quality and/or MT output quality is lower than a generic free online service.

Lastly, raw MT output should be processed before a post-editing job for automatic removal of empty and/or untranslated segments and duplications, fixing of diacritics, punctuation marks, extra spaces, noise, and numbers; terminology should also be checked for consistency and a spellcheck should be run. A post-processing stage should also be envisaged involving encoding, normalization, formatting (especially tag injection,) a terminology check and, obviously, a spellcheck.

=======================

Luigi Muzii has been in the "translation business" since 1982 and has been a business consultant since 2002, in the translation and localization industry through his firm. He focuses on helping customers choose and implement best-suited technologies and redesign their business processes for the greatest effectiveness of translation and localization related work.

This link provides access to his other blog posts.

Post Editing - What does it REALLY mean?

While many people may consider that all post-editing is the same, there are definitely variations that are worth a closer look. This is a guest post by Mats Dannewitz Linder that digs into three very specific PEMT scenarios that a translator might view quite differently. Mats has a more translator-specific perspective and as the author of the Trados Studio manual, I think provides a greater sensitivity to the issues that do matter to translators. 

From my perspective as a technology guy, this post is quite enlightening as it provides real substance and insight on why there have been communication difficulties between MT developers and translator editors. PEMT can be quite a range of different editor experiences as Mats describes here, and if we now factor in the change that Adaptive MT can have, we now have even more variations on the final PEMT user experience.  

I think a case can be made for both major cases of PEMT that I see from my vantage post, the batch chunk mode and the interactive TU inside the CAT mode.  Batch approaches can make it easier to do multiple corrections in a single search and replace action, but interactive CAT interfaces may be preferred by many editors who have very developed skills in a preferred CAT tool. Adaptive MT, I think, is a blend of both and thus I continue to feel that it is especially well suited for any PEMT scenario as described in this post. The kind of linguistic work done for very large data sets is quite different and focuses on correcting high-frequency word patterns in bulk data, described in this post: The Evolution in Corpus Analysis Tools. This is not PEMT as we describe here, but it is linguistic work that would be considered high value for eCommerce, customer support and service content and any kind of customer review data that has become the mainstay of MT implementations today. 

For those in the US, I wish you a Happy Thanksgiving holiday this week, and I hope that you enjoy your family time. I have pointed out previously, however, that for the indigenous people of the Americas, Thanksgiving is hardly a reason to celebrate.“Thanksgiving” has become a time of mourning for many Native People, hopefully, this changes, but it can only change when at least a few recognize the historical reality and strive to alter it in small and sincere ways.

The emphasis and images below are all my doing so please do not blame Mats for them.

==========

I have read – and also listened to – many articles and presentations and even dissertations on post-editing of machine translation (PEMT), and strangely, very few of them have made a clear distinction between the editing of a complete, pre-translated document and the editing of machine-translated segments during interactive translation in a CAT tool. In fact, in many of them, it seems as if the authors are primarily thinking of the latter. Furthermore, most descriptions or definitions of “post-editing” do not even seem to take into account any such distinction. All the more reason, then, to welcome the following definition in ISO 17100, Translation services – Requirements for translation services:

      post-edit

      edit and correct machine translation output

Note: This definition means that the post-editor will edit output automatically generated by a machine translation engine. It does not refer to a situation where a translator sees and uses a suggestion from a machine translation engine within a CAT (computer-aided translation) tool.

And yet… in ISO 18587, Translation services – Post-editing of machine translation output – Requirements, we are once again back in the uncertain state: the above note has been removed, and there are no clues as to whether the standard makes any difference between the two ways of producing the target text to be edited.

This may be reasonable in view of the fact that the requirements on the “post-editor” arguably are the same in both cases. Still, that does not mean that the situation and conditions for the translator are the same, nor that the client – in most cases a translation agency, or language service provider (LSP) – see them as the same. In fact, when I ask translation agencies whether they see the work done during interactive translation using MT as being post-editing, they tell me that it’s not.

But why should this matter, you may ask. And it really may not, as witness the point of view taken by the authors of ISO 18587 – that is, it may not matter to the quality of the work performed or the results achieved. But it matters a great deal to the translator doing the work. Basically, there are three possible job scenarios:

1. Scenario A:- The job consists of editing (“post-editing”) a complete document which has been machine-translated; the source document is attached. The editor (usually an experienced translator) can reasonably assess the quality of the translation and based on that make an offer. The assessment includes the time s/he believes it will take, including any necessary adaptation of the source and target texts for handling in a CAT tool.
2. Scenario B:- The job is very much like a normal translation in a CAT tool except that in addition to, or instead of, an accompanying TM the translator is assigned an MT engine by the client (normally a translation agency). Usually, a pre-analysis showing the possible MT (and TM) matches is also provided. The translator is furthermore told that the compensation will be based on a post-analysis of the edited file and depend on how much use has been made of the MT (and, as the case may be, the TM) suggestions. Still, it is not possible for the translator either to assess the time required or the final payment. Also, s/he does not know how the post-analysis is made so the final compensation will be based on trust.
3. Scenario C:- The job is completely like a normal translation in a CAT tool, and the compensation is based on the translator’s offer (word price or packet price); a TM and a customary TM matches analysis may be involved (with the common adjustment of segment prices). However, the translator can also – on his or her own accord – use MT; depending on the need for confidentiality it may be an in-house engine using only the translator’s own TMs; or it may be online engines with confidentiality guaranteed; or it may be less (but still reasonably) confidential online engines. Whatever the case, the translator stands to win some time thanks to the MT resources without having to lower his or her pricing.

In addition to this, there are differences between scenarios A and B in how the work is done. For instance, in A you can use Find & replace to make changes in all target segments; not so in B (unless you start by pre-translating the whole text using MT) – but there you may have some assistance by various other functions offered by the CAT tool and also by using Regular expressions. And if it’s a big job, it might be worthwhile, in scenario A, to create a TM based on the texts and then redo the translation using that TM plus any suitable CAT tool features (and regex).

Theoretically possibly, but practically not

There is also the difference between “full” and “light” post-editing: Briefly, the former means that the resulting text is comprehensible and accurate, but the editor need not – in fact, should not – strive for a much “better” text than that, and should use as much of the raw MT version as possible. The purpose is to produce a reasonably adequate text with relatively little effort. The latter situation means that the result should be of “human” translation quality. (Interestingly, though, there are conflicting views on this: some sources say that stylistic perfection is not expected and that clients actually do not expect the result to be comparable to “human” translation.) Of course these categories are only the end-points on a continuous scale; it is difficult to objectively test that a PEMT text fulfils the criteria of one or the other (is the light version really not above the target level? is the full version really up to the requirements?), even if such criteria are defined in ISO 18587 (and other places).

Furthermore, all jobs involving “light-edit” quality is likely to be avoided by most translators 

Source: Common Sense Advisory

These categories mainly come into play in scenario A; I don’t believe any translation agency will be asking for anything but the “full” quality in scenario B. Furthermore, all jobs involving “light” quality is likely to be avoided by most translators. Not only does it go against the grain of everything a translator finds joy in doing, i.e. the best job possible; experience also shows that all the many decisions that have to be made regarding which changes need to be made and which not often take so much time that the total effort with “light” quality editing is not much less than that with “full” quality.

Furthermore, there are some interesting research results as to the efforts involved, insights which may be of help to the would-be editor. It seems that editing medium quality MT (in all scenarios) takes more effort than editing poor ones – this is cognitively more demanding than discarding and rewriting the text. Also, the amount of effort needed to detect an error and decide how to correct it may be greater than the rewriting itself and reordering words and correcting mistranslated words takes the longest time of all. Furthermore, it seems that post-editors differ more in terms of actual PE time than in the number of edits they make. Interestingly, it also seems that translators leave more errors in TM-matched segments than in MT-matched ones. And the mistakes are of different kinds.

These facts, plus the fact that MT quality today is taking great steps forward (not least thanks to the fast development of neural MT, even taking into account the hype factor), are likely to speed up the current trend, which according to Arle Lommel, senior analyst at CSA Research and an expert in the field, can be described thus:

"A major shift right now is that post-editing is being replaced by “augmented translation.” In this view, language professionals don't correct MT, but instead, use it as a resource alongside TM and terminology. This means that buyers will increasingly just look for translation, rather than distinguishing between machine and human translation. They will just buy “translation” and the expectation will be that MT will be used if it makes sense. The MT component of this approach is already visible in tools from Lilt, SDL, and others, but we're still in the early days of this change."

In addition, this will probably mean that we can do away with the “post-editing” misnomer – editing is editing, regardless of whether the suggestion presented in the CAT tool interface comes from a TM or an MT engine. Therefore, the term “post-editing” should be reserved only for the very specific case in scenario A, otherwise, the concept will be meaningless. This view is taken in, for instance, the contributions by a post-editor educator and an experienced post-editor in the recently published book Machine Translation – What Language Professionals Need to Know (edited by Jörg Porsiel and published by BDÜ Fachverlag).

Thus it seems that eventually we will be left with mainly scenarios B and C – which leaves the matter, for translators, of how to come to grips with B. This is a new situation which is likely to take time and discussions to arrive at a solution (or solutions) palatable to everyone involved. Meanwhile, we translators should aim to make the best possible use of scenario C. MT is here and will not go away even if some people would wish it to.

-------------

Mats Dannewitz Linder has been a freelance translator, writer and editor for the last 40 years alongside other occupations, IT standardization among others. He has degrees in computer science and languages and is currently studying national economics and political science. He is the author of the acclaimed Trados Studio Manual and for the last few years has been studying machine translation from the translator’s point of view, an endeavour which has resulted in several articles for the Swedish Association of Translators as well as an overview of Trados Studio apps/plugins for machine translation. He is self-employed at Nattskift Konsult.