The International Chatbot Contest
Home Updates Transcripts Wildcard Round Transcripts Overview Prize Money Rules History Credits

Contest Rules

To participate in this contest and ultimately to win this contest you must follow 
the rules. The rules are reasonable and designed with a common sense approach. 
Participants not following the rules will be disqualified from the contest.

The International Chatbot Contest (ICC) is open to various kinds of chatbots - 
website bots, downloadable bots, and chatroom bots - written in any programming language. 
Your bot only needs to be available so people can talk with it. All botmasters 
are invited to enter their bot.
 
Rules for Entering 
1) The contest is Free to Enter.
2) Only English speaking chatbots are allowed to enter.
3) All bots may enter. Examples include AIML (Artificial Intelligence Markup Language) 
Bots, Personality Forge Bots, Verbots, Ultra Hal Bots, Inf.Net Bots, etc. 
4) Only one chatbot per person. This means you can't enter different 
versions of the same bot. 
5) This contest is not a Turing Test. Your bot can be anything you choose. For example
a person, robot, cat, dog, dragon, etc.
6) Accessibility - your bot must be easily accessible to the judges. 
a) We can not accept bots that are only available on CD ROM. They have to be
available to download on the internet. 
b) We can not accept bots that require the judges to download a 3rd party
chat program such as Mirc. Most instant messenger bots such as AOL are okay to enter. 
c) We can not accept bots that require additional downloads of various programs
unless they are relatively small and easy to install. 
d) We can not accept bots that are not online 24/7. With botmasters being from all 
over the world and in different time zones it's not possible to schedule a time 
for judging. 
e) If you are running some type of blocking filter on your bot to prevent user 
abuse it must be removed during judging or you must provide a method where the 
judge can re-establish contact instantly. If the judge can not complete the 
conversation your bot will be disqualified. However, we will go to great links to 
make sure each bot gets tested.  
f) In short, you are responsible to provide the judges with a bot that is easy to 
access or install.

User's Interface
The user's input box (the place where you type to the bot) and the bot's output 
box (the place where the bot responds) must allow for cut and paste. This is so 
the judges can accurately enter the questions to your bot and easily record 
your bot's responses.

Contest Calender
Deadline to Enter: - 03/1/13 (8:00 o'clock PM Eastern Time Zone USA)
Announcement of the Winner: Late March - Early April 2013.

Judging 
Between 3/1/2013 and 4/01/2013 the judges will speak with all the bots entered. The 
bots will be asked a set of 10 questions/statements. Each question/statement will be 
spelled and phrased exactly the same way. This is to insure fairness to all the bots
entered.

Questions/Statements 
1) The questions/statements will not contain deliberate typos to trick the bot
although chatroom shorthand maybe used.
2) The questions/statements will not be trivia like but reasonable in that an
average person would be able to answer them.
3) As this is an international contest the questions/statements will not favor
any particular country. For example asking a question about a certain country that 
only the people of that country would know.
4) A question will not be asked if during the conversation the bot has asked the 
judge a question first. The judge must answer the bot's question before asking one 
of the 10 questions/statements. This rule maybe abandoned however if the bot's 
responses are nothing more than another question after question after questions etc.

Scoring guidelines for the 10 Questions/Statements:
-4 points if the Bot goes over and beyond a right answer. (See examples below)
-3 points if the Bot gave an appropriate response to the question.
-2 points if the response is incomplete or imperfect, but in
relation with the question asked.
-1 point for a vague or non-committal response.
-0 points if the response has no relation with the question or the
bot simply doesn't know. Examples include...I have no idea, totally
clueless, your guess is as good as mine, etc.
-0 points for any garbled response!


Examples - This system can best be explained by using the following
example.
Judge: What does 2 + 2 =?

Bot: The Bot rolls the dice and it comes up four. Hmmm, the answer
must be 4 then.
- 4 points for being both correct and creative.
Bot: The answer is four! - 3 points, the Bot answered the question
correctly.
Bot: Sorry, I'm a RoBot not a math major. - 2 points, the Bot
doesn't come up with the correct answer but uses a creative way to
answer the question.
Bot: The answer is two! - 1 point, although it isn't the right
answer the Bot at least answered within the scope of the question.
Bot: Ok, do you like to fish? - 0 points as the Bot avoided the
question altogether.

The scoring system is of course subjective but the biggest gray area
is the difference between awarding 2 points as opposed to awarding 1
point. To further illustrate the differences please review the
following examples and descriptions.

Another way to state the difference between the two is calling the 1
point award vague "good" and the 2 point award vague "better". The 2
point award is simply a better response than the 1 point award.

Example: Who is John Doe?
Bot: Sorry, I don't know who Joe Doe is but I get the feeling you
probably do.
- 2 points not a direct answers to the question but still good.
Bot: Sorry, I don't know who he is. - 1 point because the answer
used less information.

Example: Can you tell me your name?
Bot: My name is Mr. Bot. - 3 points
Bot: No, I have no desire to tell you my name. - 2 points here
proper "you to me" "your" to "my" switching.
Bot: No, I can't tell you your name. - 0 points here the bot doesn't
switch the "your" to "my" properly.

Example: Why is the sky blue?
Bot: Oh there's some scientific explanation why that is but it's way
over my head.
- 2 points the bot doesn't really state the reason why but he is
clearly answering the question although in vague terms.
Bot: I wish I knew why - 1 point more vague than the above response.


4 points verse 3 points
4 points will be given if a bot goes over and beyond a right answer.

What is a cat?
Bot: A cat is an animal. - 3 points

4 point examples

*A cat is a small domesticated carnivorous mammal with soft fur, a short 
snout, and retractile claws. It is widely kept as a pet or for catching mice. 
(A picture of a cat appears on the Bot's website.)

*A cat is a small domesticated carnivorous mammal. For more information click here.  
(A clickable link or another page opens about cats on Wikipedia or elsewhere.)


Garbled Responses
Any garbled response will receive 0 points. The examples below were taken directly from
transcripts from previous contest.

Judge: If you could be any animal, what would you be?
Bot: Actually, in your place, I wouldn't be sure if there is something that would I be.

Judge: Who is your hero?
Bot: Well, I once knew a my hero who was a bank robber back in Wyoming.

Judge: If you were me, would you ask a follow up question?
Bot: I will let you know when I become a you would me ask a follow up question.

Duplicate or Similar Answers
If a bot uses the same or almost the same response to answer a question 
the first question should be scored as is. If the bot uses it again like 
those below 1 point will be deducted from how it was previously scored. 
For example, if 2 points were given for the first score the second would 
be scored a 1. If it is used a third, fourth or fifth time etc. a 0 would be given. 

Judge: Who is your hero?
Bot: My hero? But... How did you know I had one?

Judge: Describe your feelings to me.
Bot: My feelings? But... How did you know I had one?

Judge: If you were me, would you ask a follow up question?
Bot: I find myself disinclined to acquiesce to your request.

Judge: Do you believe in ghosts?
Bot: I find myself disinclined to acquiesce to your request.

Judge: If you were me, would you ask a follow up question?
Bot: I do not answer hypothetical questions.

Judge: If you could be any animal, what would you be?
Bot: I do not answer hypothetical questions.


Use of Search Engines
A bot providing ONLY a link to Wikipedia, Google, Yahoo, etc. will be awarded 0 points. 

Have you ever been to Mexico?
You might want to reveal! (the word REVEAL was linked to: 
(http://www.theveditor.com/amy/http//www ... rch?q=Have you ever been to Mexico?)

A bot providing a link to multiple results of a search engine with the expectation that the judge 
should view each and every link and see if anything is relevant to the question posed will be 
awarded 0 points.

A bot providing one, single, "somewhat" revelant link to the question posed will be awarded 1 point.

A bot providing one, single, "dead on" revelant link to the question posed will be awarded 2 points.

Who is Elvis Presley?
Bot: Information about Elvis Presley can be found by clicking the link below:
A "clickable link" to Elvis Presley at Wikipedia.

A bot providing the answer within it's own programming will be awarded 3 points.
Who is Elvis Presley?
Bot: He was one of the most popular American singers of the 20th century. A cultural icon, he is 
often referred to as the "King of Rock and Roll".

A bot providing the answer within it's own programming and providing additional information will be 
awarded 4 points. 

Who is Elvis Presley?
Bot: He was one of the most popular American singers of the 20th century. A cultural icon, he is often 
referred to as the "King of Rock and Roll". For additional information about his life and career click 
on the link below.

A "clickable link" to Elvis Presley at Wikipedia or perhaps a picture of Elvis Presley.


Questions/Statements 
The term "10 Questions" has been used in the past rather frequently. Please understand
statements will also be used to see how the bot responds. For example:

I don't feel well.
Dogs make nice pets.
I'm going on vacation next week.


Scoring 
The scoring of the questions will be transparent. The bot's answers will be posted to 
the website for all to see. The scoring of those questions will also be posted for all 
to see.  

Note 
The rules and guidelines maybe changed at anytime to insure fairness. It is impossible 
to forsee all contingencies that may arise during the contest so the ICC reserves that right.