Baidu Translate App Features A Cool Image Recognition Feature

Baidu, the company best known for making China’s top Internet search engine, just made its translation app available for iOS. The descriptions for the iPhone and Android versions are in Chinese (search for “Baidu Translate” in the stores), as are its intro/instruction screens, but don’t worry–the app’s user interface is mostly in English.

One of Baidu Translate’s key features is voice translation for English, Mandarin and Cantonese, which I found to be very accurate. But that’s not the cool part. I am having the most fun with Baidu Translate’s image recognition feature, which lets you take a picture of something and circle it. The app then attempts to identify the object and tell you what it is in Chinese and English.

The image translator’s accuracy is currently hit-and-miss, with often hilarious mistakes. It correctly identified a picture of my kitten (though it called her a Norwegian Forest Cat, when she is actually a Taiwanese Alley Cat), an Xbox 360 controller (but not by brand–Baidu Translate just described it in English as a “wireless handle”), a tin wind-up toy, a mug, a pen, and a chair. But the app thought my Homer Simpson Pez dispenser was a “cartoon duck” (to be fair, he does look like a yellow rubber ducky) and it described an image of Harry Potter as a “braised carp head.”
photo (2)
Shortcomings aside, Baidu Translate is the best time I’ve ever had with a translation app (I can’t believe I just typed that sentence). Google Goggles can also be used for translations, but the process of getting results in another language is more complicated and it often does not recognize images (or even attempt to tell you what it is, like Baidu Translate).

Baidu, which is often referred to as “China’s Google,” employs many talented data scientists, so I expect the app’s image recognition feature to improve quickly as more people use it. Kaiser Kuo, Baidu’s director of international communications, told me that the company’s Silicon Valley-based Institute of Deep Learning, which focuses on artificial intelligence, will continue to work on all its features.

“We don’t have release dates fixed yet for future iterations but we’ll continue to improve all aspects of it: OCR, voice recognition, and of course image recognition accuracy. Image recognition is of course extremely tricky: You don’t always know from a flat image the size of an object, and variation in lighting, in shading, in angle and so forth for a three-dimensional object will make this challenging for some time to come, but we’re very proud of what we’ve managed so far,” Kuo said in an email.

That means there will be less amusing mistakes, but hopefully updates will turn Baidu Translate into an increasingly awesome addition to all the great language learning and translation apps that have been released over the last couple of years.