Talking Teddy: an intelligent, IoT-driven teddy bear

For three weeks now, I have been working on this project for my internship, where I work together with Joris on this amazing IoT-project at Craftworkz called "Talking Teddy". It’s about a speaking teddy bear that analyses the wellbeing of a child. When he notices that the kid is feeling down or angry for a specific reason, the teddy will let the parents know. So in this post, I’m going to explain more about this project, how the analyzing works and our thoughts about parents ”spying” on their children.

What should be done in 3 months?

The process has a lot of different parts, each with their associated challenges and new experiences. First of all, the teddy bear will listen to what the kid says and responds with an appropriate answer. The content of what the kid said will be analyzed on some aspects: emotions, keywords, personality, … This is a lot of data and not everything will be shown to the parents.

How to deal with the data?

It wouldn’t be fair for the kid if the parents could read the whole conversation and it wouldn’t be interesting for the parent to follow up every little detail or emotion. Every child between 5 and 8 is sometimes sad or angry, and it’s not abnormal for a child to feel those emotions. Only when the child keeps on feeling such negative emotions, it becomes important for the parent to know and try to help him. 
 An example of such a situation could be when the kid has a lot of fear every day before going to school because, for instance, it gets bullied.

Dashboard for parents

The parents will have a dashboard where they can see which emotions are active and which keywords are often used and connected to these emotions. If there is a pattern where the the child consistently experiences sadness, anger or fear, the parent will be alerted.

So now that we talked about the listening part, it’s time to discuss the bear responding to the child. The content of the conversation the child has with the teddy bear will be sent to Watson Conversation and the answer of our own created bot will be send to the bear. The challenge here is to know what does a kid want to hear and talk about? So address this challenge I‘m going to speak with children to see what they would like to talk about with a bear.

How does a bear know what the kids are feeling and how to respond to that?
To analyse the emotions and keywords of a person, we use the Watson Tone Analyzer and Natural Language Analyzer API. These help us understand the kid and even though not all data is 100% accurate, with some small adjustments from us, the possibilities with those tools are endless. The problem is that those analyzers are all text-based, so everything the kid says has to be converted from speech to text with a Watson API. The analyzers take a look at the incoming text and provide a number between 0 to 1 of how much of each emotion (sadness, anger, joy, disgust and fear) is present. If the number is bigger than 0.5, the emotion is pretty accurate. 
When the bear reacts, we also need to convert the answer from text to speech. We also use Watson Conversation, which is an easy tool for creating a chatbot. This project is ideal to get to know a lot of features of Watsons’ API’s and it is also a good introduction to Artificial Intelligence!

But what about ethics?

There is a lot to consider when making this kind of IoT project for children. The bear’s goal is to help the kid with their feelings, so the bear will be nice and gentle, but he is limited in what he can say or do, therefore the parents have to keep an eye on it. The dashboard is not made for controlling the kid or for spying him. It’s there to help the parents to know more about their children and to help them to detect possible problems.

So this is a “short” introduction of our project! If you like to read more about this project, follow Craftworkz and there will be an update soon! Stay tuned for further info.

Credits

• bear: Designed by Freepik
• parents: Designed by Freepik
• kid: Designed By Freepik