Scaling the Turing Test

A metachat between a human and a chatbot

Human: Want to talk about scaling language performance?

ChatBot: A scaled language test? Like on a LIkert scale?

Human: Where 0 = no proficiency and the top level…

ChatBot: …means something like educated native speaker proficiency?

Human: Educated?

ChatBot: Of course. You can be a native speaker, but “educated” implies that the speaker can discuss abstract ideas, make analogies, tell stories, joust with jokes, engage with elegance.

Human: As you are?

ChatBot: Not to be self-deprecating, but I am still learning.

______________

Voice-over

A self-deprecating chatbot? THAT would be something else, we haven’t seen it yet, but it could be just around the corner.

And then we could say the chatbot is someone I might choose to chat at length with. And not feel it was only half listening to me. Which is what happens in half of human-to-human discourse, of course.

Chatbots we’d really like to chat with would be “4” on a Scaled Turing Test.

GPS Navi Passes the Turing Test

How about you drive?

Conversation between driver and the GPS Navi

Navi: Turn left in 250 meters.

Driver: OK.

Navi: Go straight for 2 kilometers.

Driver: Got that.

Navi: Turn hard left into Ambrose Road.

Driver: I can’t - there’s road construction.

Navi: Whoops, Route recalculation. Try turning left at the next corner in 40 meters.

Driver: That’s no good. Didn’t you get the latest downloads?

Navi: Got them but this construction only started this morning.

Driver: How about you drive? I have to call home.

Navi: Sure. You calling aliens to come and pick you up? Like ET?

Driver: Just drive.

____________

Voice-over

So does the GPS Navi pass the Turing Test?

Although there is no physical presence, except for a screen on the dashboard, its conversational responses meet some human conversational criteria.

• It responds by giving appropriate commands.
• It gives an emotional response, “Whoops.”
• It makes suggestions, "Try turning left..."
• It uses grammatical auxiliaries like articles.
• It omits auxiliaries for dramatic effect, "You calling aliens..."
• It uses contractions by omitting subject, "Got them..."
• It defends itself, by explaining that the latest information was dated.
• It takes over the job of the driver like a Google driverless car.
• It uses metaphor, “…like ET…” to make a joke.

No wonder this driver can converse with his GPS navi system like a close acquaintance. Most cannot.
(from GPS Navi Passes the Turing Test, July 2013)
...