Not All Of This Ga. Tech Professor’s TAs Are Human

Artificial Intelligence

AI or artificial intelligence is already in use in the form of self-driving cars and as part of many advanced web programs, including Google searches.

AI is about making computers smarter and more human – giving machines tools to reason and respond on their own.

“Artificial intelligence looks at human intelligence and builds machines that work in the same way as human intelligence and then it looks at artificial intelligence, intelligent machines and uses that to understand human intelligence,” said Georgia Tech computer science professor Ashok Goel.

Goel has 13 human graduate student TAs in his class on artificial intelligence. They help answer student questions in an online discussion forum.

In the spring of 2016, Goel introduced an AI teaching assistant who he calls Jill Watson, to post announcements and automatically answer frequently asked questions. Jill Watson is a reference to the former CEO of IBM, Thomas J. Watson.

Student Engagement

Goel said the goal was to save time for the human TAs, who could now focus on more difficult questions, but also to keep students engaged.

“If a student asks a question and does not get an answer for, let’s say, 24 hours, because no one is available to give an answer, the student has moved on to something else by then,” Goel said. “On the other hand, an AI can give an answer almost immediately; then the student is still engaged with that topic.”

The mystery of whether it was a human or a computer responding to a student’s question also increased engagement.

Class Mystery

In the fall of 2016, Goel had two artificial TAs. Swapnal Acharya is a first-year master’s student specializing in interactive intelligence at Georgia Tech.

“I knew that there are going to be a couple of AI TAs into this class,” Acharya said. “So I was, from beginning, I was looking for signs.”

For her classmate Christopher Cassion, it was like a game. He incorrectly thought one of the human TAs, Jose Delgado, was not real.  

“At some point Jose posted something, and I saw a mistake, and I was like ‘That’s an AI!’ I saw the name at the bottom and it said Jose and I was like ‘Oh! Maybe Not,’” Cassion said. “You’re so busy with school, but from time to time when you sit back and you’re on the class forum, it’s kind of fun trying to guess who’s who.”

Delgado was the lead teaching assistant and the only human TA the students had met.

Meta AI

The TA said the students were all part of an experiment on artificial intelligence on multiple levels.

“This is an artificial intelligence class where students design artificial intelligence as part of the class,” Delgado said. “They’re talking about it, and they’re interacting with AI that they’re talking about at the same time, so that creates this sense of actually being part of the development instead of just learning on the side.”  

Most students couldn’t tell the difference between the humans and computers. Only a few, like Christopher Cassion, correctly guessed the identity of both artificial TAs. He was tipped off on one of them when it quoted material from a previous semester.

“His reply was correct, but he quoted something that was nowhere to be found,” Cassion said. “I don’t know if the quote came up by mistake or he generated the answer, but I took it as he was trying to quote something that wasn’t available this semester.”  

Emotional Intelligence

Another human TA in the course, Will Hancock, pointed out that’s just one of its weaknesses.

“If a student falls in a hole on the street one day and can’t turn an assignment in, the [artificial] TA may not know how to handle that situation,” Hancock said.  

First-year PhD student Katie Cunningham said the AI teaching assistants were great at answering questions, but incomplete in motivating students.  

“For AI in general, I think there’s thinking about people’s emotional state as well as just getting like content and data delivered,” Cunningham said. “That’s the human side that I think AI can’t quite do at this point.”

Jill Watson 3.0

And for now, that’s good news for Hancock and other human TAs.  

“If human TAs are completely made redundant, then society will change in a way that we cannot even fathom right now,” Hancock said. “That is the domain of science fiction and pure speculation and guess at this point, because that means we have fully cognitive agents that can achieve tasks that a person can.

“And, you know, what will that look like?”