The 2008 Loebner Prize - awarded yearly to the chatbot that comes closest to passing a restricted Turing test - has been awarded to Elbot. Elbot, created by Fred Roberts of Artificial Solutions, managed to fool 3 of the 12 judges which, at 25%, is the best performance in the history of the prize. I recommend chatting to the bot yourself for a while because you'll soon see that, frankly, it's pretty damn bad: it gives generic answers to specific questions, tries (and fails) to be funny, reeks of canned responses, has very limited conversational memory, has almost no general knowledge and is very easily confused. Elbot, in other words, is all artificial and no intelligence.It's important to note that the Loebner prize is in fact very controversial and widely dismissed in the academic community. Indeed, many critics think that its design is flawed and Marvin Minsky has called it "obnoxious and stupid". Even the congenial Dan Dennett has criticized the prize harshly.
Incidentally, I hadn't previously read the paper, "Computing Machinery and Intelligence," in which Turing sets out his celebrated test. I have to point out that portions of it are deeply silly, especially (but by no means solely) the bit where Turing endorses ESP...
(See also: New Scientist's piece).
That's useful - thanks!
ReplyDeleteRespectfully speaking, I think it is unfair to say Elbot has "no intelligence". I would say, by definition, it is 25% intelligent enough to pass for a human, which, if one subscribes to the intentionalist stance of human intelligence, as I do, isn't too bad. Also, if I may say so, Marvin Minsky did a lot to set back AI with his attack on Frank Rosenblatt of the Rosenblatt Perceptron in his attack tome, 'The Perceptron'. So I don't think he should be commenting on AI, after what he did to Rosenblatt.
ReplyDeleteWell, by definition, it is indistinguishable from 3 particular humans under highly restricted conditions... That it's 25% as intelligent as a human simply does not follow. I recommend Dennett's criticism of the prize...
ReplyDeleteI don't know about Minsky's attack on Rosenblatt, so I can't comment.