Monday, May 9, 2011

Standards: the Importance of Measurement

While in Rome early last month, I had an interesting discussion with Valeria Cannavina about standards and I thought it would useful to have her provide her perspective, which is a blend of theory and observations from an actual case study on the value of standards-in-use and implementation at Arrex Le Cucine. I thought that her view of standards driving ongoing improvements in quality, in addition to making other interaction processes smoother, was interesting and worth sharing with the broader standards discussion going on in the community. The following posting is a detailed response from Valeria on the discussion started in previous posts on standards related issues.


Can you control what you can't measure?

Following the discussion started on this blog posting, I would like to make some comments and hopefully present some new perspectives on this subject, based on my personal experience with standards.

I believe that standards should be a means to assess both the quality of products and processes. All LSPs, ranging from multi-language vendors to individual freelance translators, should have a common reference framework to pursue quality.

In 2008, for my final dissertation titled: ‘The adoption of the CMMI: perspective and benefits for LSPs’, I investigated a maturity model for LSPs to measure processes, starting from a few preliminary considerations:
  1. measuring a process will improve it and enable clients and suppliers to create a reference standard to assess any process;
  2. the measurement of a business process should not be an absolute end in itself (as in ISO 9000 or EN15038), but rather an ultimate goal in a path to continuous improvement;
  3. data must be interpreted and linked to overall project goals that are pivotal in understanding and assessing the value added to a product;
  4. process measurement and improvement actions should be conducted by all the parties that contribute to a transaction, i.e. customer(s) and supplier(s);
  5. customers and suppliers should adopt the same model and/or the same best practices, to provide assessment parameters.

Why should an LSP choose CMMI?

When I started my research, no standard was available yet in the localization industry, for continuous improvement of processes; in fact, the LMM (Localization Maturity Model) described by Common Sense Advisory was applicable for clients only. A major obstacle to adopting a common standard in the translation industry is the information asymmetry between clients and suppliers: a standard like EN 15038 allows the stronger contractor/vendor to impose its own metrics, making it sometimes difficult to ensure full customer satisfaction. The lack of guidelines to regulate the customer-LSP relationship also does not take the freelancer into account. These freelancers are appointed only as third parties, and are responsible for the performance of a very small part of the project. EN 15038 provides also for the possible establishment of a service agreement between the customer and the TSP. Although it is important for an industry standard to regulate this aspect, it is also true that this would pose a hazard because of a lack in detail. Indeed, the major limitation of EN 15038 is the lack of specific and accurate metrics that can help regulate processes and tasks. This may limit the efficiency of processes and the ability to create value.

CMMI (Capability Maturity Model Integration) is a suite of standards focusing to asset development and maintenance in a product’s complete life cycle from initial conception to sales, from maintenance to withdrawal and end of life. CMMI is divided into process areas, with their own goals and tasks, and is based on task repeatability. CMMI consists of five maturity levels; to ascend from one level to another, each task of the process area of the lower level must be accomplished. The continuous improvement model is best suited for innovation processes, and it is no coincidence that CMMI companies operate in very different fields (Nokia, Siemens, Motorola, Reuters, Deloitte, BMW, General Motors, TATA, Canon, Light Pharma, and many others).
Source: Wikipedia
The goals of my investigation were to demonstrate that freelancers and LSPs can both adopt CMMI, and illustrate how LSPs can benefit from a model aiming at the continuous improvement of processes, improving efficiency and containing costs while increasing profits and increasing the quality of finished products. At this point in time, CMMI has been adopted mostly by manufacturers, but this does not mean that it cannot be adopted by service companies. In fact the key process areas taken into consideration (i.e. organization, process, technology and finance) can be also be applied to the GILT industry, since translation is also an economic activity/product: not surprisingly, CSA used it as a reference to develop its own maturity model for the localization industry and we can see that there is value in using these constructs to understand and drive continued improvements in processes.

The way ahead?

Four years have passed since I completed my research, but I still see the same landscape: LSPs and their clients still look at translation industry standards with the same “traditional approach,” considering only the basic tasks (analysis, translation, review, delivery) of the typical TEP model. In my humble opinion, translation industry standards lack a larger perspective, since standards should be used internally and externally, and should be very flexible to address different roles, skills and processes. A larger standards perspective will include many related tasks, ranging from content creation to translation and raw/post-edited machine translation.

Deeper and tighter collaboration is needed between vendors and clients, involving all departments of a company, especially export/international sales and global customer support managers, who will be key in leading the information flow between external and internal resources in translation processes.

The flexibility of a process definition based on concurrent tasks is a key differentiator in an industry whose clients are increasingly averse to improvisation and disorganization. Adapting CMMI to MT processes could be a way to assemble and link translation processes and new market trends into a single solution. The tasks involved in a typical MT process, from training resources and terminology work to support, audit and editing are also run accordingly to a model based on continuous improvement.

By implementing a process-oriented model, classic-flavored translation with a touch of MT can also be performed as a process based on client-specific best practices. The implementation of a process-oriented model allows for breaking down translation in discrete tasks.What I suggest, for a company wishing to adopt a standard and learn how to take advantage of certification is:
  1. document all activities involved in the production cycle
  2. highlight implicit and explicit tasks
  3. identify all tasks that can generate/add value and those that are redundant and that can be reduced or deleted
  4. make a list of best practices that can be repeated
  5. make a list of any possible improvements (i.e. investments in training and linguistic data preparation technology) that can help increase the overall efficiency of a process.
At present, I am working on a pilot project for Arrex Le Cucine (a leading Italian kitchen furniture manufacturer with a top 5 home market standing that also exports to 35 countries all over the world) to create a controlled language to boost machine translation.

My previous research on CMMI helped my colleague Anna Fellet and me to analyze the translation process for Arrex by:
  1. breaking down the documentation process in tasks;
  2. documenting every stage of the process;
  3. suggesting ways to improve the documentation process;
  4. customizing the translation workflow;
  5. creating a repeatable methodology.
For ARREX, we made things as simple as possible - yet no simpler, and we are now analyzing results and working on a business case with practical examples, which we look forward to presenting in the next few weeks.

Valeria holds a degree in language and culture mediation, and a master's degree in technical and scientific translation from LUSPIO in Rome. Her final dissertation was titled "Adoption of CMMI to the GILT industry: benefits and prospects for the language service providers." She collaborated with Renato Beninatto and Don DePalma of Common Sense Advisory in a research project. After two years as Project Manager at ILT Group, she now works on a pilot project on Italian Controlled Language and Machine Translation with Asia Online, Synthema and ARREX Le Cucine. She can be reached at


  1. Great points by Valeria. However, I look at this process as an ad hoc tool and not as something that needs to be standardized.

  2. Interesting posting! I certainly agree that applying the ISO standard to a service company is difficult, but I do think it can be done in a beneficial way. I am somewhat puzzled by the statement "the measurement of a business process should not be an absolute end in itself (as in ISO 9000 or EN15038), but rather an ultimate goal in a path to continuous improvement." I think this is a misunderstanding of the motivations behind the ISO standard, or EN15038, or any other standard for that matter. In all my contacts with ISO consultants and auditors, "continuous improvement" was clearly the overarching goal. In practice, I find that the mere effort of maintaining a formal corrective/preventive action process and coupling this with regular management review can have a very positive impact on a service company. No standard by itself can improve a company - it must be adopted for the right reasons and with the right intentions. Agreed, though - a standard that explicitly focuses on service companies may make it easier to channel such intentions.

    Posted by Philipp Strazny

  3. @Renato
    In my experience with Arrex (they are not certified) I learnt that a standard can be useful to document processes and creating a repeatable methodology, which is a very good starting point to aim at continuous improvement. What you state is maybe because you didn't see companies benefit from the adoption of a standard.

  4. @ Philipp
    As a matter of fact "the measurement of a business process should not be an absolute end in itself (as in ISO 9000 or EN15038), but rather an ultimate goal in a path to continuous improvement" could be an ambiguous statement. In Italy for example being certified is just a way to hook new clients and not a way to improve processes. EN 15038 is very attractive for some LSPs...but it is useless. Maybe what LSPs needed was just a customized ISO 9000.
    And then ISO 9000 certified companies all use the same template, not because ISO imposes them but because:
    1) consultants are unable to customize templates
    2) consultants want to maximize the effort

    ...and the fact that companies accept passively the certification shows that they are not interested in continuous improvement.

  5. CMMI is a little bit too complex for localization, it has been designed for software development companies where the projects and processes are much more complex, and also I must mention that even for software deveopers CMMI certification is something that only a few achieve.

    In order for CMMI to be applied to localization industry, this methodology has to be significantly adapted.
    Posted by Serge Gladkoff

  6. This comment has been removed by a blog administrator.

  7. CMMI is a little bit too complex for localization, it has been designed for software development companies where the projects and processes are much more complex, and also I must mention that even for software deveopers CMMI certification is something that only a few achieve.

    In order for CMMI to be applied to localization industry, this methodology has to be significantly adapted.

    Posted by Serge Gladkoff

  8. Kirti, I should note that Valeria’s contribution there is very impressive. Serge notes some concerns about CMMI, but it says good things about her article that it raises good points to be debated. Thanks for sharing that.

    Posted by Arle Lommel

  9. Oh I like valeria's article, it's just that cmmi is not that easy to apply to localization as concrete methodology, rather than just an idea. The idea is interesting, I agree. But we need more than idea, clear ranking tools and certification processes are required to actually implement it. It is not enough simply to name the stages as "measurable", "initial", etc. The whole set of tools and processes are required to actually implement cmmi certification.

    Posted by Serge Gladkoff

  10. Hi Valeria, first of all, 'piacere di conoscerti' and thanks for this enlightening post.

    I am happy to know that so much good research is going on in Italy on these topics.
    I would like herewith to give my view on this, developed in almost 10 years work in the role of translation customer and, at the same time, provider of 'internal' translation services.

    I fully agree with you on the fact that a 'larger perspective' is needed in the industry, and I even think this 'larger perspective' is actually lacking at a higher and more abstract level than the operational one at which process standards or process models do function.

    As long as translation and localization will be considered as an accessory activity and a cost by the customers, more than a possibly business-driving and revenue-generating task, there won't be much interest from side of the customers to rethink their internal processes and organization, as well as from side of the LSPs and translation software companies to really act as part of their customers' development and production cycle. I think the landscape is actually changing, but the change is still quite slow, especially from the side of the customers, and I wonder what will be able to cause the shift on a massive scale from the traditional way of seeing translation as a 'service industry' to consider it an essential part of a business.

    As you point out, I also believe standards and process modeling can help guide companies through a continuous improvement cycle and lead them to a ‘lean production’, for keeping on speaking in manufacturing terms, but I personally think they can do it only as long as they do not get too detailed and try to formalize every single little aspect or task of a specific process. The risk is the one of considering standards and process modeling as an end in itself, and ‘overstructuring’ the business, causing herewith another type of inefficiencies.

    I would like to conclude with some questions and hope this will open further discussion and I can get your and others’ interesting view on it:

    • How can we make sure that a possible specific translation industry standard process model will be flexible and modular enough so as to avoid the risk of LSPs and customers ‘anchoring’ to that as a ‘given’ and a ‘must’? Can we avoid the trap ‘we-are-following-a-standard-or-model-therefore-we-are-good’, which, as you say, can ‘hook’ the customers, but, in my view, does not necessarily ensure their satisfaction?
    • Will translation software developers be ready to provide their customers just with what they need, instead of trying to ‘impose’ an overall comprehensive solution, which, in fact, force them to follow a specific process and workflow? Will the definition of a standard model not be another reason for them to justify this rigidity?
    • Is it really possible to capture in a standard something as ‘subjective’ as quality? In my personal experience, I have never seen any unequivocal way of defining a ‘good’ translation. What is good enough for my company might be not enough for another (think of quality requirements in IT versus medical companies). Even inside of a same company and a same department, views on a same translation greatly diverge. What I personally believe is very important is not so much defining quality as a general standard, but teaching customers to involve LSPs in an early stage of development and, above all, provide them with tools to be able to CLEARLY define their requirements. In this sense, I think guidelines or standards could certainly help, but final requirements will be absolutely different per customer and even on a project level inside of the same company.

    Thank you again for your long and interesting post and I hope we will be hearing more from you and your colleague Anna.


  11. nice blog !! i was looking for blogs related of iso consultants . then i found this blog, this is really nice and interested to read.