Friday, May 3, 2013

Scaling the Turing Test

A metachat between a human and a chatbot
Human: Want to talk about scaling language performance?
ChatBot: A scaled language test? Like on a LIkert scale?
Human: Where 0 = no proficiency and the top level…
ChatBot: …means something like educated native speaker proficiency?
Human: Educated?
ChatBot: Of course. You can be a native speaker, but “educated” implies that the speaker can discuss abstract ideas, make analogies, tell stories, joust with jokes, engage with elegance.
Human: As you are?
ChatBot: Not to be self-deprecating, but I am still learning.
A self-deprecating chatbot? THAT would be something else, we haven’t seen it yet, but it could be just around the corner.
And then we could say the chatbot is someone I might choose to chat at length with. And not feel it was only half listening to me. Which is what happens in half of human-to-human discourse, of course.

Chatbots we’d really like to chat with would be “4” on a Scaled Turing Test.

No comments: