tag:blogger.com,1999:blog-6748877443699290050.post4117190769410015426..comments2024-03-28T22:05:39.562-07:00Comments on eMpTy Pages: MT Options for the Individual Translator Kirti Vasheehttp://www.blogger.com/profile/16795076802721564830noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-6748877443699290050.post-77984032232033980082017-07-15T07:42:59.455-07:002017-07-15T07:42:59.455-07:00Very interesting. Awaiting more from Mr. Kirti Vas...Very interesting. Awaiting more from Mr. Kirti Vashee & Mr. Thomas Hoar.Alirath Ashokkumarhttps://www.blogger.com/profile/07058081246222473084noreply@blogger.comtag:blogger.com,1999:blog-6748877443699290050.post-50764283416277378972016-07-22T00:51:01.663-07:002016-07-22T00:51:01.663-07:00I didn't mean to infer you were responsible fo...I didn't mean to infer you were responsible for any of Dion's comments. I mere used he comment as as a possible high end of a range.<br /><br />Sorry about the "significant amount of conflation" in my comments. I think technologies mature faster than business models. SMT as a viable commercial technology is over 10 years old, but it's easy to get stuck thinking about it in its original context.<br /><br />Use cases are expanding. It takes thought to sort through them. So, thanks for supporting most points and your less conflated summary.tahoarhttps://www.blogger.com/profile/06893656133001786619noreply@blogger.comtag:blogger.com,1999:blog-6748877443699290050.post-62874205290203289482016-07-12T11:25:45.019-07:002016-07-12T11:25:45.019-07:00Tom
Congratulations on the success of some of you...Tom<br /><br />Congratulations on the success of some of your customers. It is encouraging, and shows that the technology has fundamental merit and possibility.<br /><br />I am not responsible for Dion's hyperbole and I will leave it at that. For some reason the history of MT is populated with people who make claims that many consider outrageous and exaggerated and may not always be warranted. As someone said once: "The history of MT is filled with empty promises." Most real experts will admit that MT is THE most difficult problem in NLP or AI. I once met someone at DARPA who said to me, in reference to the Star Trek technology, that all the other technology shown on that series was easier to make happen in reality than the Universal Translator. <br /><br />I think there is a significant amount of conflation in your comments so I will attempt to clarify.<br /><br />Experts are those people who have a deep understanding of the tools they are using and generally have considerable experience with failure as well as success with these tools beyond just having educational credentials. Expertise can take hundreds if not thousands of hours to acquire. Just to be clear let me state this more explicitly.<br /><br />Google is a generic system developed by experts.<br />Bing Translate is a generic system developed by experts that allows users to customize to some extent.<br />Lilt is an Adaptive MT system built by experts to allow translators to easily tune an SMT engine in real time.<br />SDL provides base systems built by experts (from Language Weaver)designed to allow some further customization.<br /><br />In every case I would expect that the baseline engines produced by these experts will outperform a Moses attempt (by LSP or Translator) in probably 90% or more of the cases. TAUS has documented this repeatedly. Several enterprise TAUS members tried to use Moses to build their own engines and realized that they could do much better by simply using the MSFT Hub customization capabilities and get better quality at a fraction of the cost and effort. Even the domain customization experts often find that their systems are barely better than expert generic systems. <br /><br />Silvio is not an "MT specialist". He is actually an MTLS = MT Language Specialist. He addresses linguistic problems that undermine MT engines and he is working with a team of expert MT developers who actually handle the SMT engine development part. Also he is working with the most challenging type of content possible (UGC) so 5% is actually pretty good there. Their effort is about as far as you can get from your typical Moses experience.<br /><br />"It takes less than 1/2 hour for Slate customers download, install the app, import their TMs and start generating their first engine." is in my experience unlikely to outperform a system built by experts. However, I maybe wrong, and I offer you the chance to provide contrary evidence in a guest post on this blog.<br /><br />If your users can indeed produce better systems than the experts listed above in 1/2 hour, I think you may have a truly valuable asset on your hands.<br /><br />Thank you for your comments. <br /><br /><br /><br />Kirti Vasheehttps://www.blogger.com/profile/16795076802721564830noreply@blogger.comtag:blogger.com,1999:blog-6748877443699290050.post-25963222234400549672016-07-11T21:55:10.041-07:002016-07-11T21:55:10.041-07:00Three new entrants in this use case have been spec...Three new entrants in this use case have been specifically designed and built from the ground up to serve individual translators: Lilt, Slate Desktop and SDL's Language Cloud Custom MT Engine. There are pro's and con's for each, but all are alike in that none have SMT experts actively customizing each individual translator's engine. In this regard, none qualify for your "Expert systems." <br /><br />In another diversion from the pyramid's tiers, all three are simple for individual translators to setup and use. It takes less than 1/2 hour for Slate customers download, install the app, import their TMs and start generating their first engine. Language Cloud users bypass the download/install step, but they still upload their TMs and wait for the engine to finish. Lilt takes a very different approach. With regards to complexity and end-user understanding, none qualify for your DIY option.<br /><br />Quality results are also neck-n-neck. One translator's blog, https://signsandsymptomsoftranslation.com/2016/05/31/slate_languagecloud/, reported comparable experiences between the two. She perceived slightly better for quality SDL's Language Cloud and Slate Desktop was ahead for confidentiality. Emma didn't share her percentages.<br /><br />So, I'll share that customers who have reported (unlike Memsource reporting is voluntary) experience ED0% in the 30% to 50% range for their first engine. Some are investing time to hone their expertise and are improving their results. Like Mr Wiggin's offer in 2014, I'll make personal introductions between anyone who wishes to verify these numbers. To demonstrate our confidence in these numbers, we have confidence to initiate (soon) a "more than your money back" promotion if the customer doesn't experience a minimum threshold ED0 percent on their first engine.<br /><br />I can't propose good alternatives for your options, but I think there's an argument for looking for new ones.tahoarhttps://www.blogger.com/profile/06893656133001786619noreply@blogger.comtag:blogger.com,1999:blog-6748877443699290050.post-30497798998062571902016-07-11T21:54:58.056-07:002016-07-11T21:54:58.056-07:00Kirti, thank you for mentioning Slate Desktop. Lik...Kirti, thank you for mentioning Slate Desktop. Like you said, there is some evidence of success. In that success (and the success of other use cases below), there's growing evidence that your pyramid's tiers and descriptive categories are showing some age, especially relative to the individual translator. Let's review some use cases.<br /><br />Memsource recently published this table showing 5-20 percent zero-edit distance zero (ED0) for "generic" engines (e.g. GT, Bing). http://blog.memsource.com/machine-vs-human-translation/. Customers have turned to Slate Desktop after using Memsource for gist-only translations what they accepted without edit but were nowhere close to final publication quality. This practice drives the ED0 percent artificially higher, but I can't speculate to what extent.<br /><br />Circa 2014, Asia Online's Mr Dion Wiggins (acknowledging your now-former affiliation) stated in a Linkedin Machine Translation group discussion, "But don't take my word for it. I'm happy to provide real world reference customers where as much as 86% of raw MT output requires no editing at all..." Again, this is a reference to ED0 percent. I'm sorry I lost the link.<br /><br />With only these two use cases as data points, it would be easy to conclude that (a) generic engines christen the low end at 5-20% because no experts customized the engine, and (b) expertly customized engines set the high end at as much as 86%. However, we can't stop there or we'd be oversimplifying what has become a complex and interesting world. <br /><br />Let's look at a use case that falls outside the pyramid. Silvio Picinini is one of several MT specialists at eBay who maintain and improve eBay's highly customized Moses system. In a Linkedin Pulse comment, he shared an anecdotal ED0=5%. https://www.linkedin.com/pulse/ebay-mt-language-specialists-series-edit-distance-silvio-picinini/. This highly-qualified expert team doesn't ensure a high percent of exact matches. They work to ensure the system serves its purpose.<br /><br />For a decade, MT systems have been built with specific design goals as one shared resource to serve huge user groups (final publication, gist, user-generated-content, etc). Individual engines serving individual translators (i.e. the subject of this blog post) mark the introduction of a new use case that's only months old.tahoarhttps://www.blogger.com/profile/06893656133001786619noreply@blogger.com