Loebner Prize facts for kids

Hugh Loebner, who started the competition

The Loebner Prize was a yearly contest in artificial intelligence (AI). It gave awards to computer programs that seemed the most human-like to judges. The competition used a special test called the Turing test. In this test, a human judge would chat by typing with both a computer program and a real person at the same time. The judge then tried to guess which one was the computer and which was the human.

Hugh Loebner started this contest in 1990. It was first held with the Cambridge Center for Behavioral Studies in Massachusetts, USA. Later, in 2004 and 2005, it took place in Hugh Loebner's apartment in New York City. Some experts in artificial intelligence didn't like the Loebner Prize. One famous critic, Marvin Minsky, said it was just for publicity and didn't really help AI. From 2014, the AISB organized the prize at Bletchley Park in the UK. Other places like Flinders University, Dartmouth College, and the Science Museum in London were also involved.

For its last competition in 2019, the rules changed. There were no special judges. Instead, the public judged the chatbots, and there were no human competitors. The Loebner Prize stopped completely in 2020.

Prizes for Human-Like Programs

At first, the prize for the most human-like program was $2,000. This amount changed over the years. For example, it was $3,000 in 2005 and $2,250 in 2006. In 2008, the winner received $3,000.

There were also two very big prizes that were never won. A $25,000 prize was offered for the first program that judges could not tell apart from a real human. This program also had to convince the judges that the human was actually the computer! An even bigger prize of $100,000 was for a program that could pass a Turing test that included understanding text, pictures, and sounds. The competition was supposed to end once this huge prize was won.

How the Competition Worked

The rules for the Loebner Prize changed a bit over the years. In the early days, conversations were limited to certain topics. But since 1995, people could talk about anything with the programs.

In 2007, some simple "screening questions" were used to check the programs. These included questions about the time or general knowledge, like "What is a hammer for?". They also asked questions to see if the program could remember earlier parts of the chat. All words used in the questions came from a dictionary for children or teenagers under 12.

In 2008, for the first time, programs from the internet could join. Many different people were invited to be "interrogators" or judges. These judges had limited time to chat. In 2003, they had only 5 minutes per program. From 2004 to 2007, they had more time, over 20 minutes per pair (one human, one program). From 2008 to 2009, judges had 5 minutes to chat with both the human and the program at the same time. This time increased to 25 minutes from 2010 onwards.

Why Some People Criticized the Prize

Many experts in the field of AI did not like the Loebner Prize. They had several reasons for this.

Some thought it was just a way to get attention, not a serious scientific competition. Marvin Minsky even joked about offering a prize to stop the competition!

The rules sometimes made judges make quick decisions. Early on, judges had very little time to ask questions, sometimes only 2.5 minutes. Also, conversations were sometimes limited to one topic chosen by the program. This meant programs could use simple tricks to seem smart.

Critics also felt that programs in the competition didn't try to truly understand or be intelligent. Instead, they used basic tricks, like those seen in early chatbot programs such as ELIZA. Programs that were good at tricking people often won.

Past Competitions and Winners

Here are some details about a few of the past Loebner Prize contests:

2003 Competition Highlights

In 2003, the contest was held at the University of Surrey in the UK. No computer program managed to fool the judges into thinking it was human. The winner was a program called Jabberwock, made by Juergen Pirner. Elbot came in second, and Jabberwacky was third.

2006 Competition Highlights

The 2006 contest took place at University College London. The judges included a professor of cybernetics and a professor of artificial intelligence. The winner was 'Joan', which was based on Jabberwacky, both created by Rollo Carpenter.

2007 Competition Highlights

The 2007 competition was held in New York City. Again, no program passed the Turing test. The judges ranked the programs:

• 1st: Ultra Hal, made by Robert Medeksza
• 2nd: Cletus, a private entry by Noah Duncan
• 3rd: Jabberwacky, by Rollo Carpenter

The winner received $2,250 and a medal. The other two received $250 each.

2008 Competition Highlights

The 2008 competition was held at the University of Reading in the UK. Over one hundred judges tested programs in an early phase. Five finalists competed. Elbot won the bronze award for being the "most human-like." It fooled three out of twelve judges into thinking it was human. This was very close to the 30% success rate often needed to say a program passed the Turing test. Two other programs, Eugene Goostman and Ultra Hal, each fooled one judge.

2009 Competition Highlights

The 2009 Loebner Prize was held in Brighton, UK. The prize money was $3,000. The top three programs were by David Levy, Rollo Carpenter, and Mohan Embar. A writer named Brian Christian was a human participant in this competition and wrote about his experience.

2010 Competition Highlights

The 2010 competition was the 20th time the contest was held. It took place at California State University, Los Angeles. The winner was Bruce Wilcox with his program, Suzette.

2011 Competition Highlights

The 2011 competition was held at the University of Exeter in the UK. The prize was $4,000. The top four programs were Rosette (Bruce Wilcox), Zoe (Adeena Mignogna), Chip Vivant (Mohan Embar), and Tutor (Ron Lee). That year, there was also a panel of younger judges. Their results were different, with Tutor and Zoe tying for first place.

2012 Competition Highlights

The 2012 competition was held at Bletchley Park in England. This was to celebrate 100 years since Alan Turing was born. The prize was $5,000. The top programs were Chip Vivant (Mohan Embar), Angela (Bruce Wilcox), Adam (Daniel Burke), and Linguo (M. Allan). For the first time, the conversations were shown live online.

2013 Competition Highlights

The 2013 competition was held in Northern Ireland. The top programs were Mitsuku (Steve Worswick), Tutor (Dr. Ron C. Lee), Rose (Bruce Wilcox), and Izar (Brian Rigsby). There was also a Junior Loebner Prize, where Mitsuku and Tutor tied for first place.

2014 Competition Highlights

The 2014 competition was back at Bletchley Park. It was filmed live by Sky News. The winner was 'Rose' by Bruce Wilcox, who received $4,000 and a bronze medal.

2015 Competition Highlights

In 2015, 'Rose' by Bruce Wilcox won the Loebner Prize again.

2016 Competition Highlights

The 2016 Loebner Prize was held at Bletchley Park. The winners were:

• 1st place: Mitsuku
• 2nd place: Tutor
• 3rd place: Rose

2017 Competition Highlights

The 2017 Loebner Prize was also at Bletchley Park. This year, a new way of sending messages was used. The winners were announced by a Nao_(robot) (a small robot):

• 1st place: Mitsuku
• 2nd place: Midge
• 3rd place: Uberbot
• 4th place: Rose

2018 Competition Highlights

The 2018 Loebner Prize was the last one held at Bletchley Park and in its traditional Turing Test style. The winners were:

• 1st place: Mitsuku
• 2nd place: Tutor
• 3rd place: Colombina
• 4th place: Uberbot

2019 Competition Highlights

The final Loebner Prize in 2019 was held at the University of Swansea. The contest changed a lot. It was part of a bigger event about computer creativity. Instead of a few judges, members of the public, including schoolchildren, could chat with the programs. They knew the programs were not human. Seventeen programs took part. Steve Worswick won for a record fifth time with Mitsuku, which put him in the Guinness Book of Records.

A special jury also voted for their favorite programs: Most humanlike chatbot:

• 1st place: Mitsuku – 24 points
• 2nd place: Uberbot – 6 points
• 3rd place: Anna – 5 points

Best overall chatbot:

• 1st place: Mitsuku – 19 points
• 2nd place: Uberbot – 5 points
• 3rd place: Arckon – 4 points

Winners

Here is a list of the official winners of the Loebner Prize:

See also

In Spanish: Premio Loebner para niños

• List of computer science awards
• Artificial intelligence
• Glossary of artificial intelligence
• Robot
• Artificial general intelligence
• Confederate effect
• Computer game bot Turing Test