Baidu scientists admit to ‘mistake’ on supercomputer test amid cheating claims
PUBLISHED : Thursday, 04 June, 2015, 3:27pm
UPDATED : Thursday, 04 June, 2015, 4:12pm
Stephen Chen [email protected]
Baidu's reputation has been tarnished by claims of cheating in a supercomputer test. Photo: Reuters
A group of scientists from Chinese search engine giant Baidu have apologised amid claims they cheated on a complex image-recognition test, which led to their being banned from similar events by the organiser for a year.
“We apologise for this mistake and we are continuing to review the results,” said Dr Ren Wu, leader of the Baidu Heterogeneous Computing team, in a letter to the organiser and the research community released on Tuesday.
The scandal, which reportedly saw the group’s supercomputer fend off high-profile challenges from the likes of Microsoft, has shaken Baidu’s credibility and reputation both at home and abroad.
The company said earlier that their artificial-intelligence technology was ahead of international competitors like Google.
“Our company is now leading the race in computer intelligence,” Ren said at a tech summit in California last month. “We have great power in our hands – much greater than our competitors.”
The team claimed that, with the backing of the Chinese government, they were building the world’s largest and most sophisticated AI platform for research and commercial applications.
The ImageNet large scale visual recognition competition (ILSVRC) is one of the world’s biggest AI competitions organised each year by Stanford University, the University of North Carolina at Chapel Hill and the University of Michigan.
It requires a computer to classify objects from a set of 100,000 random images into 1,000 different categories.
This year, the ILSVRC organisers allowed each team to access the test data base twice a week during the span of their project in order to improve the performance of the image-recognition system.
But it was later discovered that, by using many different accounts, the Baidu team had accessed the test data pool over 200 times in six months.
At its peak, it had made over 40 entries in five days, or more than 20 times the official limit.
In a paper documenting the test, Baidu researchers said their supercomputer Minwa recorded an error rate of 4.58 per cent, beating the 4.82 per cent reported for a Google computer in March and Microsoft’s earlier 4.9 per cent.
The organiser later issued a statement on their website explaining why the ban had been enforced.
It said it had found that competitors could gain a “potentially significant advantage” by “exploiting the ability to test many slightly different solutions on the test server.”
A researcher with Tsinghua University in Beijing who specialises in artificial intelligence said the Baidu team had schooled their supercomputer to think like a test-cracking Chinese student.
“The more exercises it did, the higher it scored in the final exam,” said the academic, who declined to be named due to the close ties between Baidu and the university.
“But everything was done just to gain a higher score. It doesn’t mean the computer is smarter than other machines."
"Similarly, Chinese students who learn how to ace tests are not necessarily any better than their foreign peers at solving problems in real life.”
