Prototyping an AI presentation assistant: from idea to solution

Some time ago, we were invited for a meeting with an international chemicals group with an interest in AI. The conversation started pretty similar to most our client meetings (paraphrasing): “We’ve heard about AI, but we’re unsure of the possibilities”. As an AI service company, we’re used to starting our sales pitches from this position, but what came next surprised us:

“So why don’t you build us something to convince us?”

This “shoot first, ask questions later” approach is exactly how companies should deal with innovative technologies like Artificial Intelligence.

Now, those of you who know about our heritage, know that Brainjar started out as a spin-off of a prototyping company called Craftworkz, so proving the value of new technologies in a cost- and time-effective manner is a challenge we love to tackle.

So, after some brief introductions, we started thinking about how we could be of service. Obviously, for chemical companies, there are a lot of opportunities regarding the use of predictive analytics to improve quality and reduce downtime in production lines. However, before we could help we would first need to gain some domain knowledge in order to build an effective solution, which is suboptimal for a first project with its main goal of demonstrating what AI is capable of. Eventually, we landed on the idea of building an application that uses “out-of-the-box” AI services, allowing us to demonstrate multiple domains of artificial intelligence in one single project without having to scour the client's systems for relevant training data.

The case

After a while, we honed in on a somewhat surprising use-case: as an international chemical company, the client obviously has to comply with a lot of rules and safety procedures. New employees are kept up-to-date with these regulations by watching multiple instruction videos in which how much attention they pay could literally mean the difference between life and death! Therefore, the final idea was to create a “presentation assistant” that uses AI techniques to evaluate a speech and give targeted feedback.

What makes this use-case ideal as an AI demonstrator is that it uses algorithms from different domains of artificial intelligence. You see, in order to evaluate a speech, you not only need to focus on the content of the speech, but also on how it is brought and on how the speaker presents him/herself. Therefore, we need algorithms that can process images, audio and text.

How it works

Below you will find a screenshot of what the application looks like. Because we are dealing with an international company, we designed a web-based application which employees can easily use from almost any location and any device.

Users are able to upload a video of themselves (or anybody really) giving a speech, and the application then takes the video through a processing pipeline, which you can see below:

This application is all about using AI components to extract structured data from media files. Each of these components analyses a different aspect of the speech, generates numbers that humans can understand, and then finally uses these numbers and manually defined rules to give natural language feedback to the employees.

Take, for example, our emotions detection algorithms: these take in a picture, audio or text, and output a series of numbers between 0 and 1 for a series of emotions (Anger, Sadness, Happiness, etc.). It is then up to humans to determine the cutoff where certain emotions are inappropriate. Some amount of cheer is of course perfectly fine, but you don’t want a serious technical video to contain large amounts of emotion.

If you are interested in learning more, please check out Dries’ blog where he outlines our approach in more technical detail.

Conclusion

We, of course, would like to thank our client for the opportunity and the wonderful cooperation thus far. However, at least for us, this is far from the final step. Now that we have gotten to know each other, we hope to continue our cooperation to the point where our AI applications become so crucial to the competitive advantage of our clients that we are no longer allowed to communicate about them! And of course, we already have some killer ideas ;-).

Are you interested in what Brainjar could mean for your business? Please don’t hesitate and contact us via info@brainjar.ai!