U of T research contributes to instant verbal translation

When Microsoft’s chief research officer took to the stage in Tianjin, China, last month, his language skills made an auditorium of hundreds of native Mandarin speakers applaud wildly.

But he only spoke English at the conference. As he explained new developments in instantaneous translation and interpretation, Rick Rashid’s English spoken words were simultaneously translated robotically to Mandarin.

The stunned crowd applauded the new computer system that can recognize human speech patterns, thanks to a University of Toronto research team.

Blogging about his experience this month, Rashid cited U of T research in deep neural networks. The research dates to 2009, when a team identified patterns in human brain behaviour, and came up with acoustic models that discriminate speech much more accurately than ever before.

“This means that rather than having one word in four or five incorrect, now the error rate is one word in seven or eight,” Rashid wrote. “While still far from perfect, this is the most dramatic change in accuracy since (a study in) 1979.”

Microsoft’s latest software translates instantaneously and tailors the user’s voice to resemble the user. George Dahl is one of the main U of T researchers that Microsoft kept on after a 2009 internship.

“It’s very gratifying to see work that my colleagues have done is out there,” Dahl said.

The technology works by programming a computer to understand basic patterns in human thought and speech through a series of algorithms. The computer is then fed hours of recorded and typed speech and makes connections faster and more accurately.

“You build a general framework and then add in specific information,” Dahl said. “You can train the computer.”

But don’t expect your smartphone to replace a human interpreter. Dorothy Charbonneau, director of conference interpreting with the Association of Translators and Interpreters of Ontario, says machine translation can be useful for short, simple phrases, but it’s no match for trained people.

“I don’t see it replacing human interpreters,” she said. “Language interpretation is not just the words; it includes tone, body language, context and jargon.”

While translators and interpreters have faced other technical advances such as Babel Fish and Google Translate, Charbonneau says she hasn’t heard any serious discussion among her colleagues of robots replacing human interpreters.

“A machine can’t translate the meaning of phrases like ‘The spirit is willing but the flesh is weak,’” Charbonneau said. “In the same way, a machine won’t pick up on puns or idiomatic expressions.”