By: Michael Feldman

An impressive demonstration of an artificial intelligence system that can debate humans was witnessed by an audience at a recent IBM event in San Francisco.

The system, called Project Debater, was developed over the past six years by a team of IBM researchers Israel.  It represents what is probably the most advanced AI system in a new and the growing research area, known as computational argumentation and debating.

In its first public demonstration held during an event at IBM’s Watson West site in San Francisco, Project Debater was instructed to argue in favor of the proposition: “We should subsidize space exploration.” According to a blog penned by IBM Research director Arvind Krishna, here is what happened:

“Project Debater made an opening argument that supported the statement with facts, including the points that space exploration benefits human kind because it can help advance scientific discoveries and it inspires young people to think beyond themselves. Noa Ovadia, the 2016 Israeli national debate champion, opposed the statement, arguing that there are better applications for government subsidies, including subsidies for scientific research here on Earth. After listening to Noa’s argument, Project Debater delivered a rebuttal speech, countering with the view that potential technological and economic benefits from space exploration outweigh other government spending.”

For an AI system, delivering an opening argument seems fairly straightforward, given that it’s essentially a recitation of the most pertinent facts surrounding a topic. But the ability to provide a rebuttal against a skilled debater would seem to demand a good deal more sophistication. For starters, it requires the AI system to pick apart its counterpart’s argument and respond to the issues he or she raised, and do so in a logical manner. That could only be done with a deep capability in natural language, plus the ability to understand high-level concepts in order to form relevant counter-arguments.

In Krishna’s write-up of the event, after the two sides delivered their closing summaries, a poll indicated that “a majority of audience members thought Project Debater enriched their knowledge more than its human counterpart.” Although that would indicate the technology has reached a fairly high level of sophistication, IBM concedes that the software does, on occasion, make mistakes.

The demonstration was followed by a second debate between the system and Dan Zafrir, another professional Israeli debater. In this case, they argued for and against the statement: “We should increase the use of telemedicine.” No account was provided of how that debated proceeded.

Krishna says the system’s intelligence encompasses three broad areas: speech writing and delivery, listening comprehension, and, perhaps most importantly, modeling human interactions in such a way that enables the generation of logical arguments. A subset of the system’s capabilities, specifically some of the speech-to-text technology, came from IBM Watson. Most of the AI components, however, including argument mining, stance classification and sentiment analysis, and more the advanced elements of natural language processing (NLP), were developed independently using different technologies.

For example, deep neural networks (DNNs) were used to score claims and evidence with respect to the topic of debate and to develop Project Debater’s speaking and listening skills. Long short-term memory (LSTM) networks and convolutional neural networks (CNNs) were also employed for automatic speech processing.

As far as practical applications go, there’s not much money to be made in debating people on esoteric topics. But potentially, the technology could be invaluable in areas like decision support and policy-making. To that end, IBM’s plan is to eventually commercialize the software and incorporate it into the Watson product portfolio and the company’s cloud offering. As Krishna writes: “Project Debater could be the ultimate fact-based sounding board without the bias that often comes from humans.”

Image source: IBM Research