Which search engine would win at Jeopardy?

Posted by Kate Taylor

IBM's Watson computer is widely tipped to take the crown when its appearance in Jeopardy is screened next month.

But according to Stephen Wolfram, the developer of Wolfram Alpha, most search engines could do a pretty good job.

He ran 200,000 previous Jeopardy questions past Google, Bing, Ask, Blekko, Wikipedia Search, and Yandex - but not Wolfram Alpha, as it's not designed for this type of query - and found that most gave a creditable performance.

"It shows us that the more mature search systems are getting to be remarkably similar in their raw performance — so that other aspects of user experience... are likely to become progressively more important," says Wolfram in a blog post.

Google topped the ranking, showing the correct answer somewhere on its first results page 69 percent of the time, just ahead of Ask and Bing. It threw up the answer actually in the headline or opening text of the top result 66 percent of the time; here, Bing came second and Yandex third.

This compares rather creditably with common-or-garden human beings, who average a 60 percent success rate. The top player of all time, Ken Jennings, has a success rate of 79 percent.

Wolfram contrasts the approach of IBM with that of his own Wolfram Alpha search engine. With the IBM system, he says, "The essential idea was to start with textual documents, and then to build a system to statistically match questions that are asked to answers that are represented in the documents."

Wolfram Alpha is based on a different paradigm. "The key point is that Wolfram|Alpha is not dealing with documents, or anything derived from them," he explains. "Instead, it is dealing directly with raw, precise, computable knowledge. And what’s inside it is not statistical representations of text, but actual representations of knowledge."

So what's his prediction for the televised game next month?

"IBM has built an impressive level of anticipation for their upcoming Jeopardy television event," he says. "IBM’s system certainly should be able to win."