tag:blogger.com,1999:blog-6748877443699290050.post8781089721763642249..comments2024-03-27T23:43:31.674-07:00Comments on eMpTy Pages: MT Output Quality Estimation - A Linguist's PerspectiveKirti Vasheehttp://www.blogger.com/profile/16795076802721564830noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-6748877443699290050.post-31187215820750163012016-08-18T13:27:46.386-07:002016-08-18T13:27:46.386-07:00Victor,
Thank you for your comment.
All tools ha...Victor,<br /><br />Thank you for your comment.<br /><br />All tools have the potential to be misused by unscrupulous users. But I think the approach suggested here empowers translators/editors in the following ways: <br /><br />1) For those who may choose to work on PEMT jobs by providing them a way to rapidly assess the MT output quality in a linguistically informed way,<br />2) Setup personal effort/work benchmarks (based on these scores) to understand the level of difficulty/effort involved in performing the new PEMT task. This would require keeping track of actual work performed versus scores these old jobs had. <br />3) To identify projects worth doing versus ones to avoid, independently, and without input from the agency.<br /><br />There is an effort involved in setting up your measurement and scoring system but then all you need to do is run new potential jobs through there and decide if it is worth doing or not. <br /><br />If properly done I think this gives a post-editor a reliable way to identify a "bullshit" project based solely on his linguistic knowledge and personal assessment of work previously done and measured.<br /><br />KirtiKirti Vasheehttps://www.blogger.com/profile/16795076802721564830noreply@blogger.comtag:blogger.com,1999:blog-6748877443699290050.post-31273333803499212022016-08-18T12:51:34.239-07:002016-08-18T12:51:34.239-07:00Thanks for your interest, Victor. My goal was to s...Thanks for your interest, Victor. My goal was to share an idea, a method, for something that I believe is lacking in the "industry". To be honest, I did not consider the potential implications you mention. How this idea can be used is outside of my control, as I guess the same happens with many other ideas once they reach the public, users, etc. But I'm hoping there will be also many positive things coming out of its use. Since the customization is done by linguists (i.e., someone that understands the language really well needs to create these features), post-editors should be able to quickly realize that the checks are not appropriate or that the system is rigged. I think cynical agencies, as you call them, can benefit more from getting accurate scores and metrics (and thus being able to budget properly, accept/reject projects, pay resources fairly, etc) than from creating a bad reputation for themselves. <br />It's an interesting idea; thank you for taking the time to share it.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6748877443699290050.post-61832027493055342562016-08-18T02:22:22.358-07:002016-08-18T02:22:22.358-07:00Interesting approach. However, your flexibility fo...Interesting approach. However, your flexibility for "customisation" and "personalisation" could turn out to be a double-edged sword. In large organisations like yours it can be used as intended - to make an honest assessment of the post-editing process and refine it over time. However, in another scenario in which agencies are desperately looking for translators to post-edit raw MT output, your flexibility could be used by cynical agencies to fiddle the statistics and use ostensibly "official" ratings to persuade inexperienced translators to accept hopeless post-editing tasks.<br />So although your project does show laudable intentions and interesting ideas, it is not yet enough to address the misgivings in the translator and post-editor community.Victor Dewsberyhttps://www.blogger.com/profile/18342577630994069368noreply@blogger.com