Failing test translations because of incompetent evaluators

Baran Keki wrote:

I remember one occasion where a dear colleague went overboard with sabotaging my test translation and actually gave the game away. They, inevitably, replaced every word with their synonyms when failing to find any translation mistakes and when they ran out of synonyms, they replaced lowercase letters with uppercase letters (like changing "Bank account" (source text) to "Bank Account" (target text)) they did that wherever they could in a 300 word text.
Long story short, I was able to demonstrate this to the Vendor Manager and 'pass' the test.
Not that it did me any good though... That Danish agency started cutting down on rates and introduced longer payment terms as soon as Covid started. Perhaps it's best to give such agencies a miss as others suggested.

mughwI wrote:



No idea. Company policy, maybe?

I was once asked to downgrade a perfectly good translation because the end client thought that grading it as flawless would give the translator ideas about charging more.

I refused, but who knows what they did on their end.

mughwI wrote:


For my part, I always mention the good points, as well as the clear effort the translator has made given the extremely short deadline, and lack of guidance. AFAIK, the detailed results of my review are not shared with the translator.

Sadly, because of this quirk, great collaboration opportunities are lost along the way.

Kevin Fulton wrote:



Agencies that require test translations often have dozens of tests to assess each year., each requiring both the assessment itself and reporting the result (thumbs up/down). The assessment is for the benefit of the agency, not the candidate. An agency is under no obligation to expend additional time/money on an unsuccessful applicant.

I've done a large number of evaluations over the years and I've often fought the temptation to contact the applicant: "The text has to be completely translated. Leaving sentences in the source language is never acceptable" or "You dimwit! Didn't you look at the accompanying image? The text is about an interior door, not a gate."

My advice is to move on, give no further thought to unsuccessful applications. There are lots of agencies, some with astute assessors.

Baran Keki wrote:

I'm sure there are independent, competent and impartial language assessors in major language pairs, especially where the target language is English, and I don't doubt Ice Scream's experience with those candidates as an expensive assessor.
However, the agencies do rely on their existing translators for such tests when the language pair is English to Mongolian, Slovakian, Turkish, Georgian etc. etc.
They don't usually get much work in those language pairs, and unfortunately some of those existing translators do not like competition either, if that makes sense.
I remember one occasion where a dear colleague went overboard with sabotaging my test translation and actually gave the game away. They, inevitably, replaced every word with their synonyms when failing to find any translation mistakes and when they ran out of synonyms, they replaced lowercase letters with uppercase letters (like changing "Bank account" (source text) to "Bank Account" (target text)) they did that wherever they could in a 300 word text.
Long story short, I was able to demonstrate this to the Vendor Manager and 'pass' the test.
Not that it did me any good though... That Danish agency started cutting down on rates and introduced longer payment terms as soon as Covid started. Perhaps it's best to give such agencies a miss as others suggested.

Eleftherios Kritikakis wrote:

-- Because the same is happening in large LSPs. Have you ever heard of any LSP where the founding team were all translators from different languages, and the PMs were also all translators at some point, and really understand what's going on and how to spot the good from the bad? Ever heard of any LSP having a permanent native language reviewer from every language, as employee with no conflict of interest? No? Neither have I.
But since this modus operandi has not caused them any significant problems because very few clients review the product in detail, they won't change. And why should they?

Zolboo Batbold wrote:



So translation agencies are basically at the whim of reviewers. If the reviewer can't do their job properly, the good translator doesn't get the job, and the bad translator or the one that offers the lowest rate gets the job.

Ice Scream wrote:
But are are still plenty of small agencies that do things properly. Why not put all that ranting energy into getting work from them?

Baran Keki wrote:

I'm sure there are independent, competent and impartial language assessors in major language pairs, especially where the target language is English, and I don't doubt Ice Scream's experience with those candidates as an expensive assessor.
However, the agencies do rely on their existing translators for such tests when the language pair is English to Mongolian, Slovakian, Turkish, Georgian etc. etc.
They don't usually get much work in those language pairs, and unfortunately some of those existing translators do not like competition either, if that makes sense.
I remember one occasion where a dear colleague went overboard with sabotaging my test translation and actually gave the game away. They, inevitably, replaced every word with their synonyms when failing to find any translation mistakes and when they ran out of synonyms, they replaced lowercase letters with uppercase letters (like changing "Bank account" (source text) to "Bank Account" (target text)) they did that wherever they could in a 300 word text.
Long story short, I was able to demonstrate this to the Vendor Manager and 'pass' the test.
Not that it did me any good though... That Danish agency started cutting down on rates and introduced longer payment terms as soon as Covid started. Perhaps it's best to give such agencies a miss as others suggested.

Zolboo Batbold wrote:



It's people like them who ruin the translation industry.

Denis Fesik wrote:

My experience as a reviewer of test translations (in quite a different market) has shown me talented translators are very far between and agencies need to grab hold of them if they want serious jobs done. What the HR people do to make them work for as little as possible is none of my business. My take on this is to give extra points to candidates for elusive things like my intuition telling me they have potential while the number of actual mistakes in their test trying to convince me otherwise. And still, only about 5–10% get a passing grade – on a test I've edited myself to make it easier to understand and translate. And I do write extensive reviews highlighting all or most of the mistakes in each test, which, I believe, get sent to the candidates as feedback (never had anyone trying to challenge any part of my review). I could also be sending them the test translation I did when applying for the same position as the correct translation, but I know there's never the correct one, and yes, my test translation was quite different, so much so that when a candidate submitted a test where most of the terminological decisions were just like mine, I thought someone had let him in on the comments from one of my reviews (that person never got the job though, maybe they weren't willing to pay him the money he wanted). MT tools are making people lazy, so whenever I see signs of human intervention, signs of research done on top of what dictionaries can offer, signs of effort put into shuffling parts of sentenses or streamlining grammatical inconsistencies to make the text easier to read and understand – all that reads to me as a good sign even if the formal criteria for marking the test as passed have not been met. So there, maybe the problem is market-specific (low demand for professional translation services, e. g.), or the TS needs to keep searching for the right people

Denis Fesik wrote:

My experience as a reviewer of test translations (in quite a different market) has shown me talented translators are very far between and agencies need to grab hold of them if they want serious jobs done. What the HR people do to make them work for as little as possible is none of my business. My take on this is to give extra points to candidates for elusive things like my intuition telling me they have potential while the number of actual mistakes in their test trying to convince me otherwise. And still, only about 5–10% get a passing grade – on a test I've edited myself to make it easier to understand and translate. And I do write extensive reviews highlighting all or most of the mistakes in each test, which, I believe, get sent to the candidates as feedback (never had anyone trying to challenge any part of my review). I could also be sending them the test translation I did when applying for the same position as the correct translation, but I know there's never the correct one, and yes, my test translation was quite different, so much so that when a candidate submitted a test where most of the terminological decisions were just like mine, I thought someone had let him in on the comments from one of my reviews (that person never got the job though, maybe they weren't willing to pay him the money he wanted). MT tools are making people lazy, so whenever I see signs of human intervention, signs of research done on top of what dictionaries can offer, signs of effort put into shuffling parts of sentenses or streamlining grammatical inconsistencies to make the text easier to read and understand – all that reads to me as a good sign even if the formal criteria for marking the test as passed have not been met. So there, maybe the problem is market-specific (low demand for professional translation services, e. g.), or the TS needs to keep searching for the right people