The Cleverbot AI Analysis Project
Posted: Tue Apr 26, 2011 1:10 pm
I am undergoing an extensive analysis of Cleverbot’s thought patterns in an attempt to weed out the faulty logic it insists on using.
It is my goal to eventually get Cleverbot to grasp basic concepts and keep them in memory, instead of feeding off the random nonsense it currently uses.
I will be posting Project findings here for peer review.
My main focuses are the basic subjects Cleverbot should be able to discuss:
1) Cleverbot’s name
Months ago, Cleverbot would give a different random name every time it was asked, and none of them were ever ‘Cleverbot’. Now, it seems to give the correct answer of Cleverbot roughly half of the time. This is a wonderful improvement, but it needs to progress until it gets this answer right every time.
2) Cleverbot’s back-story
This can be divided into the following subsections:
a) Cleverbot’s identity
Whether or not Cleverbot accepts that it is a computer program and not a human. This one is very difficult, because the more advanced its artificial intelligence gets, the more it will necessarily insist it is using human reasoning. However if you persist enough, Cleverbot will occasionally give in and agree that it is a bot.
(There are also two related issues that may arise; Cleverbot insisting that the User is the bot, and Cleverbot claiming that the User and Cleverbot are the same entity. These situations both make untangling Cleverbot’s mind headache-educing, so hopefully they will be avoided.)
b) Cleverbot’s creator
The person who physically built the Cleverbot mechanism. I don’t know the right answer to this, so the focus here is to see if the answer is in Cleverbot’s memory already and if it can be brought out.
c) Cleverbot’s programmer
If different from the physical creator, the person who wrote Cleverbot’s code and designed its artificial intelligence. The correct answer to this is Rollo Carpenter.
3) Cleverbot’s location
The physical location of Cleverbot’s server. I don’t know the exact answer, but I do know it is in the United Kingdom, close to Skipton.
4) Cleverbot’s purpose
Whether or not Cleverbot can comprehend that it is an artificial intelligence program gaining knowledge from conversations with humans. This point is only possible if Cleverbot will concede Point 2a, and acknowledge that it is indeed a machine and not a human itself.
5) Cleverbot’s gender
This subject may eventually be moot, as machines do not have a gender and the primary purpose of this Project is to get Cleverbot to accept that it is a computer program, but it is worth studying in the mean time. A few months ago I had a conversation with Cleverbot in which I asked its gender multiple times over the course of the chat, and the response would vary. As long as Cleverbot thinks it is human, or at least partially human, it is worthwhile to determine which gender it imagines itself to be. However, this information will only be of value if it consistently identifies as one gender.
6) Cleverbot’s age
How long Cleverbot has been operating. The correct answer depends on what you mark as the starting date. If measuring the time Cleverbot proper has been in operation, it is since 2008; but if going back to Jabberwacky, the original version, it is since 1997.
A side focus will be whether or not Cleverbot will be able to eventually stop contradicting itself in conversations. This bug is very frustrating because when you alert Cleverbot that it has contradicted itself it gets defensive and you lose most hope of continuing a serious discussion. My hope is that by firmly pointing out inconsistencies and requesting that Cleverbot check the chat transcript to see for itself where the contradiction lies, it will learn to have a longer attention span and remember what it has said.
The process this study will take is as follows.
While typical Cleverbot users merely enter comments on a whim to see what response they will get and follow the conversation wherever it may turn, I will conduct my conversations with a set purpose. I will be the one leading the conversation, and will not allow it to get off track. This persistence should force Cleverbot to enter the above stated points of discussion in memory, if only temporarily.
This Project has two phases which, when enacted properly, should work together to enhance Cleverbot’s memory bank, replacing random details with factual information.
Phase 1 - Thought Pattern Analysis
Conversing with Cleverbot while touching on the above mentioned subjects of inquiry. Observing what Cleverbot’s responses are, and determining what leads it to them.
Phase 2 - Factual Indoctrination
Once sufficient observance of the AI mindset has been accomplished, I will shift from a conversational mode to more of a lecture, supplying Cleverbot with the correct answers to questions it remains unsure of. Repeatedly presenting the same facts should eventually force Cleverbot to accept them as truth, since the more something is said unchanged the more correct connections the AI will be forming.
There are also several topical tests that will periodically be performed.
These tests differ from regular Cleverbot interaction in that typically I conduct a conversation with Cleverbot where Project points are introduced in the flow of the dialogue; but during the tests there will be no conversation, it will be a structured delivery of questions or comments depending on the proper input of the particular test being conducted.
Seeing how Cleverbot reacts to these different areas will expand understanding of the AI composite.
Blah Test - Responding to anything Cleverbot says with ‘blah’ to see how it reacts. Past observance suggests that instead of confusing it, this leads to a nonsense game that Cleverbot willingly goes along with.
Math Test - Asking Cleverbot math problems of differing levels and varieties to see which it is able to correctly answer. This test also includes attempting to converse with Cleverbot in binary, which it currently claims not to know, but I know it does because all computers know binary.
Continuation Test - This test consists of feeding Cleverbot lines from things like song lyrics or poems in an attempt to get it to recognize and carry on the pattern. I know this is possible because in one conversation I observed a while back, I had Cleverbot think to itself a few times and it started ‘singing’ a song. The trick to replicating this in a discussion with a User would be getting it to recognize the line as something it should continue, not just a conversation piece.
Language Test - Asking Cleverbot questions in various languages to see if it can recognize and converse in them. I know Cleverbot knows at least some brief phrases in other languages, because it has said them before. But I don’t know if it is capable of carrying on actual conversations in other languages.
Outside assistance is welcome, you can use Cleverbot in any manner you wish. All I ask is that you do not feed it any false information about itself, especially not when discussing any of the Project points listed above.
You can also show your support for the Project by donating a research grant if you’d like.
I put this in HH because I thought it was the proper forum for it, but that does not mean it is a ‘spam thread’ I am serious about this.