Thursday, April 21, 2016

The Concept of MT Maturity

I would like to introduce a series of posts that looks at and discusses the concept of MT Maturity. I hope to illustrate that getting real business advantage from MT requires alignment with a variety of other related business processes. I also plan to look at how the MT technology continues to evolve, especially as related to use in professional translation and corporate use to make large amounts of content multilingual. 

To facilitate the discussion I have created my own very rough evaluation and analysis model, which is an adaptation of the CSA Localization Maturity Model which is an adaptation of the software industry’s capability maturity model (CMM). Basically, it is a way of assessing if the technology user understands the technology, and is also using it in an efficient and effective manner by properly linking it to other organizational processes. 

Thus, companies that are at a higher stage of the CSA LMM model referenced above, will tend to have much more responsive and adaptive localization practices, that enable them to be much more nimble and efficient and effective in international market initiatives. These maturity models define very specific “stages” to characterize the efficiency and effectiveness of the business process as shown below. Thus an LSP at a higher stage is probably much better to work with, and an Enterprise at a higher level is also much more nimble and international market savvy and superior localization practices.

While this is interesting, it is somewhat theoretical, and I am going to attempt to make it less so in my own analysis of MT technology. This will make my comments less academically robust, but since all I do here is just speak my opinions on things it does not really matter. I do NOT represent any corporate opinion here, and these comments are purely personal observations though hopefully still valid professional assessments. My intention is to highlight best practices and point out what I think are interesting trends.

MT (machine translation) or Automated Language Translation (ALT) or Machine Pseudo Translation (MpT) use can also be described to some extent using these maturity model stages, however, I am going to try and simplify this further, as I only use the maturity model perspective to better structure my comments and analysis, of how this might apply to the business use of MT technology to further international initiatives.

So while there are still many naysayers who will never see any value to professional business use of MT, there is a growing community of users that are working through the vagaries and complexity of MT with varying degrees of success. The Gartner Hype Cycle curve below provides a useful graphic to describe the stages from a very common expectations cycle, that so many, if not most users of this technology seem to go through. I will describe the various maturity stages in more detail in upcoming posts though my analysis may not be quite as tightly defined as the CSA analysis.

It may be useful at the outset, to list some of the most common pitfalls, which are the opposite of best practices, as the continual recurrence of these factors seem to plague many business use cases of MT technology. They are:
  • Looking for cheap, fast and “easy” approaches like Moses and Instant Do-it-yourself solutions.
  • Expecting to do better than the really decent generic engines provided by Microsoft and Google without investments in time and core skill building.
  • Looking to use MT as a wholesale replacement for human translation.
  • Using MT for one-off projects and/or for small volume requirements (LT 500,000 words). MT makes most sense for very large projects that involve millions of words, especially those that would not make sense to do using only human translators for time and cost reasons.
It is worth re-stating that MT technology in the right hands and right use cases can generate long-term sustainable competitive efficiencies, but this generally needs a strategic focus, and a long term commitment to the technology deployment to achieve this.

Emerging MT Trends

While the “translation industry” only focuses on the older SMT approaches most often based around Moses, we are seeing an increasing and building momentum around newer approaches to building MT engines. These involve techniques like deep learning, neural nets and artificial intelligence to improve on the results of the current approaches. These new approaches are apparently yielding much better results than ideas like morpho-syntactic approaches to SMT which have also had small imporvements.

The presentation here presents some of the new perspectives in AI and Neural MT versus the traditional SMT NLP views in a relatively understandable way.

I have also noted that Microsoft appears to have seriously stepped up their MT technology of late. They are attempting to solve the most difficult automated translation challenges in the world today, specifically:
  • Facebook comments (so I now understand what Clio Schils, Renato and my Russian Facebook friends are talking about)
  • Skype based multilingual voice conferences
  • Customization with limited sets of training data using AI and allowing four levels of customization.
Also of interest, a new kind of interactive MT solution that I think holds great promise, especially for individual translators, are adaptive MT solutions like Lilt that take real time corrective feedback and improve dynamically while also leveraging TM in multiple ways. They are also built on post-Moses technology, so provide much better foundation engine quality which means less post-editing. While only available for a limited set of languages, I think it is one of the few MT technology initiatives that actually generate enthusiasm from translators. For an individual translator, I think that using something like Lilt is a much better approach than trying to build your own Moses engine since you have real expertise building your foundations. This article provides a good overview of what adaptive MT looks like and tries to do. 

I found out that Prince died today and while I was never a real fan, I think his guitar solo here is quite exceptional, and should leave no doubt to his amazing musical ability. Theatrics aside, it is the notes and varied textures that he plays that make it so special, and wins the respect and admiration of the other band members.  It is worth a listen. May he rest in peace.



  1. Hello Kirti,

    Thank you for this very interesting article, and the Prince guitar solo.

    I would like to know what you think about MT engines that translators can create and keep for themselves, such as Slate Desktop.


    1. Charles,

      I think these desktop custom engines are unlikely to be as robust and capable as the main generic engines, unless the user really understands what they are doing, and has a substantial volume of relevant, clean data.

      In most cases I would expect that the generic engines outperform small-scale Moses efforts, except where there is a technical and unique subject domain that relies heavily on specialized terminology.

      I would expect that something like Lilt would easily outperform a desktop Moses engine in terms of output quality and overall translator productivity. Comparatively, Lilt also has a very low startup time and short learning curve.

      Perhaps some translators could do a comparative test and show whether this is true or not.


    2. Charles, thanks for asking. I agree with Kirti that it's difficult to gauge without independent voices speaking out.

      Re "Perhaps some translators could do a comparative test and show whether..." I agree. It would be nice if some translators would speak up, but therein lies the challenge. They're a are pretty savvy group.

      Those who experience maximum benefits are choosing silence. They haven't talked to me about MT maturity models. They do talk about how many of their clients feel entitled to discounted rates from translators who benefit from better tools and training that help them work faster.

      Ask a skilled and experienced plumber to fix a broken toilet on Sunday at 3:00 AM. Then ask him for a discount just because he finished the work in less time than expected because he was prepared with good tools and inventory in his van. Do clients really respect plumbers more than translators?

    3. tahoar said: "Those who experience maximum benefits are choosing silence."

      One way to easily handle this is to submit a post or personal evaluation done anonymously. I will publish an unedited comparitive evaluation that any translator may be willing to share. This can be done in the comments section here or by emailing the comments/post.



  2. Hi, Kirti,

    Many thanks for this post and the MT Maturity.
    As I understand, with MT Maturity, you mean the implementation of MT and not the maturity of the tool itself.
    It is unfortunate that CSA labeled their maturity model **Localization** Maturity Model leaving out translation. Localization is a subset of translation. This is significant. We need to get the terminology right, otherwise communication will fail.
    As you mention in the article, when implementing MT companies need to put that in overall context of their business strategy. And companies often fail here because they lack the expertise in MT. I see this happening with agile and analytics. Everyone is speaking about these concepts, but how many people really know about that?
    So, I am looking forward to your "series of posts that looks at and discusses the concept of MT Maturity."

    Thank you.
    Ayfer Bektas