Man vs. Machine: computer takes on Jeopardy champs
Josh Visser, CTV.ca News Staff
Published Saturday, February 5, 2011 6:08AM EST
The category is Battle of Wits. The clue is: It knows nothing for certain, yet may be the greatest challenger in the history of the game show Jeopardy!
The question is elementary. What is Watson?
For three straight days starting February 14, IBM's latest computer creation, known as "Watson," will be facing two famed Jeopardy! champions, Ken Jennings and Brad Rutter.
There's only a teeny-tiny matter at stake -- can a machine be smarter than man?
Four years ago, IBM began work on Watson to follow-up the success of "Deep Blue" -- the computer that famously defeated world champion chess player Garry Kasparov in 1997.
Watson is the next logical step in evolution for computers -- IBM has attempted to build a computer that will attempt to think and learn like a person, and understand language, in a manner.
Author Stephen Baker went behind the scenes at IBM for the Watson project, writing the book, "Final Jeopardy: Man vs Machine and the Quest to Know Everything." The book follows the genesis of the project right until the big final showdown (it has been pre-taped.)
Speaking to CTV.ca, Baker says language "is the next great hurdle for computers."
While voice-recognition has been around for a while, modern day computers do not understand what a word like "operator" means. They just know which function the word serves (where to transfer a caller), but not its meaning. Watson sets out to "understand" the meaning and context of natural language -- everything from irony to riddles.
How does Watson work?
Watson "thinks" using over 100 different algorithms, each one looking at a Jeopardy! clue, and coming back with what they think is the best answer under their particular ability.
Some of the answers will be utterly ridiculous, but Watson will look at them all and figure out which algorithm has the best record given the type of clue.
"It's like evolution, each algorithm has to come back with smart answers in order to survive and be paid attention to," Baker said.
When Watson sees a clue, it knows nothing about the subject but has hundreds of millions of documents stored in its memory banks. It has to understand the language and then hunt for what is statistically most likely to be the correct answer.
Take, for example, a clue about the wife of the late Canadian prime minister, Pierre Trudeau, Baker explains.
"So it says, ‘Who is Pierre Trudeau, is he the guy who does Doonesbury, no, that's Gary Trudeau, and it brings back all these different Trudeaus, and the wife of all these different Trudeaus . . . and eventually it will get Margaret Trudeau, and start double checking," Baker says. "Is she a woman? Yes. Does she have links to Canada? Yes. And as it begins checking and rechecking, it becomes more confident in its response."
But Watson doesn't "know" Margaret Trudeau is the correct answer the way humans do, it's just highly confident that she is the most probable answer.
Strengths and weaknesses
Watson, like any other Jeopardy! player, has its strengths and weaknesses. One of its main advantages is simply that it is super fast on the buzzer.
The buzzer was actually a point of controversy, as initially Watson just had an electronic signal, but its human competitors said that would be an unfair advantage, forcing IBM to build a physical finger to press a button. But it still has the quickest trigger finger.
But unlike humans, Watson has no senses, no life experiences and has difficulty contextualizing human language. These are serious weaknesses its opponents will need to exploit.
For example, during one early test match, Watson was given a clue about 19th Century Literature during a Final Jeopardy! round and came back with the answer "What is the Pet Shop Boys?"
It's going to make a few head-desk mistakes.
To make up for that lack of contextual knowledge, Watson has a "super ability to analyze things statistically."
"It's like bats, they simulate seeing through sonar. Watson simulates understanding through statistical analysis," Baker says.
But Watson is not facing your average Jeopardy! smarty-pants. Instead, it's facing the Wayne Gretzky and Michael Jordan of the game.
Jennings won 74 games in a row (fodder for sports fans, is this a greater run than Joe DiMaggio's 56-game hit streak?) Rutter is the all-time money leader for Jeopardy! with career winnings of more than $3 million and a perfect 20-0 record.
Despite their achievements, online bookmaker Bodog.com is "sad to report" that Watson is the odds-on favourite to win the $1 million prize in the three-day match (the spoils will go to charity.)
What does Watson mean?
OK, so a computer can play a game show, big deal. What does this mean for the future?
IBM hasn't released the actual cost of Watson, but with 30 scientists devoted to the project, the cost is estimated to be in the tens of millions. Clearly, one of the top technology companies in the world believes they are on to something.
"Watson's ability to understand the meaning and context of human language, and rapidly process information to find precise answers to complex questions, holds enormous potential to transform how computers help people accomplish tasks in business and their personal lives," IBM says in a statement.
A smarter automated customer service model is the obvious advance, but IBM is looking much further, suggesting the possibility of a healthcare application, that would be able to accurately diagnose patients.
Baker suggests that we are going to come to a crossroads soon and will have to seriously re-evaluate our model of learning. What is the point of memorizing certain facts (what's the capital of Idaho? What year was the Treaty of Versailles?) when computers can tell us the answers instantly.
"We have to make decisions about what we learn and what we decide to store in our own heads," Baker said. "What is valuable? What can humans do that computers can't do going forward?"
Baker says it's not hopeless; there are areas where we mere mortals are greater than the machine.
"Watson is not creative. It has no sense of humour. No appreciation of melody. We can use machines like this to do research for us and harness them to our superior cognitive engines to come up with groundbreaking ideas in art and in science," he said.
So, the human race has a chance. Jennings and Rutter might not.
Baker's "Final Jeopardy" is already available as an e-book and the hardcover will be in stores Feb. 17. Baker promises you can pick it up for a Toronto flight and finish it by the time you land in Vancouver. He also has his own website, http://thenumerati.net/ and encourages any readers with questions to ask away and he will do his best to respond.