After delving into the realms of sports reports and movie assistants, our Generative AI Playground takes another turn. In this third article, we introduce you to our AI-powered quiz platform that transforms the traditional quiz format into an interactive, intelligent, and incredibly fun experience. The idea was to create a quiz that could be moderated by GPT, offering a dynamic way to generate questions, evaluate answers, and ensure the fun never stops. Even though this use case provides the ideal work break, it also illustrates how we can use this technology for other domains.

Smarty Party

The concept of an online quiz is not new. However, traditional systems often fall short of recognizing slightly (in)correct answers due to typos or phrasing differences, leading to user frustration. Think of the “De Slimste Mens” app, once extremely popular in Belgium. While it functioned well to a certain extent, its rigid answer validation system could unjustly reject correct answers if they didn't precisely match the expected word or sentence.

We wanted to create something similar with the added flexibility Large Language Models (LLMs) offer. We built a quiz that could understand and evaluate answers with human-like judgment, allowing for near misses or alternate expressions… Introducing: Smarty Party!

The concept of Smarty Party is simple. In our quiz app, a player can pick any category that comes to mind. The system then generates questions based on this category, according to a chosen difficulty level. Other players must then answer these questions and the AI model checks if the answers are correct (enough). Even though the concept is simple, the execution is not. One LLM wasn’t enough in this case, so we decided to develop a multi-agent GPT system.

Multi-agent GPT system

To bring Smarty Party to life, we employed a multi-agent GPT-4 system, each agent tasked with a specific role:

• The Quizmaster: Generates quiz questions based on categories chosen by users, e.g., If the category is “Apple (the company)”, the quizmaster will generate a question like “Who were the founders of Apple?”
• The Category Validator: Ensures the chosen categories make sense and are not manipulated to be inherently easy or irrelevant, e.g., The user cannot pick a category named “A category where every answer is 10” (and yes, GPT would be able to generate very creative questions for this category, but we considered this as cheating — no family fights during our quiz!)
• The Category Inspirer: When a player has no inspiration for a category, they can ask for random categories generated by this agent
• The Question Validator: Checks if the generated question inadvertently contains the answer, preventing giveaways, e.g., The following scenario cannot happen: “Where is the beer Hoegaarden brewed?” - “In Hoegaarden”
• The Answer Validator: Validates if the answer is correct or incorrect
• The Comeback Agent: Offers alternative multiple-choice questions to another player if the initial answer is wrong

This multi-agent approach is crucial, as a single GPT model cannot meet the quiz system's diverse and dynamic requirements. Each agent, powered individually by GPT-4, is fine-tuned with specific instructions (prompts), enabling specialization, adaptability, and enhanced user engagement that a single model could not achieve.

Consider the human editorial team behind "De Slimste Mens Ter Wereld," with Erik Van Looy as the quizmaster. Behind the scenes, a whole team crafts questions, validates them in real-time, and more. Just as a real-life team specializes in various tasks, so does our "AI team”.

OpenAI’s JSON mode

A technical highlight of this experiment is our use of the JSON mode of OpenAI. This mode instructs the model to always return a JSON object that makes sense for your use case by specifying this in the system message. In our case, a JSON output ensures that the output is structured in a way that can be directly used by the front-end for a seamless user experience. Working like this guarantees that the visual presentation is always consistent regardless of the query's complexity.

Beyond quizzes: endless possibilities

While "Smarty Party" focuses on creating an engaging quiz experience, the underlying technology has broad applications. This experiment highlights the power of generative AI in creating multi-agent teams that can work for any field, be it in education, human resources, or software development. To show the possibilities, we translated this experiment into some other use cases. Of course, this is a non-exhaustive list of what we could create:

• Training and e-learning: Creating e-learning modules is time-consuming and expensive. Now, automating the generation of quizzes based on textbooks or manuals is no longer a dream. Moreover, you could create a mock examiner that quizzes students in preparation for their exams or tests.
• Human resources: While interviewing candidates remains a highly human task, you could delegate some parts to an AI assistant. Think of a technical interview, where a candidate has to answer technical questions. Or, again, you could simulate job interviews to prepare recent graduates to do better in a real life.
• Quality assurance and content filtering: Ensuring generated content adheres to specific guidelines, such as excluding personal data or sensitive information. You could already use a single agent for this and prompt it correctly. However, a second GPT agent could double-check if the content really does not contain any sensitive information.
• Software development: Envision a multi-agent system where one agent writes code, another reviews it for accuracy, another writes documentation, and so forth, mimicking a full software development team. Put one human software developer at the end of the loop to check everything and you’ve created a very efficient team.

Takeaways

Smarty Party represents a leap forward in how we interact with AI for entertainment and other use cases. By pushing the boundaries of what's possible with a multi-agent set-up. We're not just creating fun and games but also exploring new ways to learn, work, play, and most importantly: make sure cheaters can’t cheat 😉. As we continue exploring different technologies in the Generative AI Playground, we're excited about the endless possibilities these technologies bring to our lives. On to the next!