<- all projects

Reviving Einstein for Vives

The challenge

“Logic will get you from A to B. Imagination will take you everywhere.” When Hogeschool Vives asked our venture nocomputer to build an interactive AI installation for their Vives experience space, the team let their imagination run wild. What if they made one of the greatest minds of all times — Albert Einstein — come back alive to educate students? Imagine asking Einstein to explain the theory of general relativity, to share his favorite meal, or to answer any question you might have. Today, you can ask Einstein’s digital twin anything — thanks to artificial intelligence.

Ainstein

The idea behind this AI-powered Einstein — also known as Ainstein — is that students should be able to experience AI first-hand. This set-up allows them to converse with a digital version of Einstein in multiple languages. What makes this digital twin stand out is its capability to respond to questions in real-time in Einstein’s characteristic manner. To make Einstein come to life, we combined these key technologies:

  1. WhisperAI: To convert speech to text, capturing user speech in (any) natural language
  2. GPT: To comprehend natural language and generate responses in Einstein’s persona
  3. ElevenLabs: To convert GPT’s textual responses into audio files using Einstein’s voice clone
  4. Audio2Face: To animate facial expressions and lip movements in sync with the generated audio
  5. Metahuman: To create a hyperrealistic visualization of Albert Einstein, modeled in detail from online photographs, including custom hair styling
  6. Unreal Engine: Renders the entire experience, providing a lifelike visual presentation

In addition to the experience, students can actually see the behind the scenes on another screen. On this screen, each step from the pipeline described above is displayed. First, they can see the speech-to-text transcription, then you see a prompt being generated for GPT, which a response is generated to, which is in turn sent to the voice cloning technology, and so on.

We invite you to step into the realm of AI with Flax, the House of AI. Immerse yourself in this unique experience and interact with one of history’s greatest minds, reborn through technology. Book your ticket here: https://www.theflax.be

Want to know more?

Related cases