Breaking Into the Black Box: An Exploration of AI and Emotion

Can an AI understand human emotion?

That’s the question we wanted to help visitors answer with the “Blackbox” interactive developed for the MIT museum. To embody MIT’s motto, “Mens et manus” (“Mind and hand), we gave visitors first-hand experience working with a neural network instead of explaining how AI works. Our goal was to break open the “black box” that is AI, making this complex technology a little less mysterious and a little more human. 

Working with our architectural partners Studio Joseph, branding agency Pentagram, and fabrication partners Kubik Maltbie, we helped integrate the experience into the built environment. 

Ultimately, three things have made this experience a hit for the MIT Museum: it’s real, it’s weird, and it’s fun. 

Making it Real: Building a Neural Network for a Museum 

For MIT, it was essential that an installation about neural networks wasn’t just illustrative or explanatory. It had to be real. 

We wanted to make something visitors could test. Something they could play with and try to break. And because the MIT Museum is about creating connections between people and technology, we wanted our neural network to focus on something human. 

So we based the project on a question: Can an AI recognize emotions in faces?

The first step to creating a neural network is training it to recognize a specific data set that it will use to construct a comparative model. In other words, before it could recognize faces, it needed to learn what faces were. Happy faces. Sad faces. Neutral faces. Angry faces. Surprised faces. So we made a simple drawing app and asked our friends, family, and the MIT museum community to sketch a series of faces that our neural network would use to build its model. 

Because this was the middle of the pandemic, and people had a lot of time on their hands, we quickly collected more than 7,000 samples of smiles, frowns, and everything in between.

Getting Weird: Neural Network Architecture meets Museum Architecture

Once the network was built, we had to figure out how to show it working. Taking something that is invisible and digital and giving it physical form in actual space is a very weird idea. Which is why we knew we had to do it. The question was how.

In early prototypes, we explored using display screens or light-up models, but neither of those sufficiently captured the sense of wonder that we wanted the exhibit to inspire. Then we found hologauze.

Hologauze is a translucent scrim designed to display 3D holographic projections. It’s an incredible technology that creates the illusion of objects floating in space. It’s a real “wow” moment that has to be seen in person to be fully appreciated. 

We designed an immersive holographic display that uses three layers of hologauze — one for each phase of the neural network process.

Layer 1: Input. The first layer shows the user’s drawing, input via touchscreen, being encoded into a grid of 28×28 pixels. Each pixel is mapped to a neuron, the fundamental unit of a neural network.

Layer 2: Compute. The second layer shows how the drawing activates the neurons in the network as it compares the user’s drawing to the model created from community drawings. 

Layer 3: Results. The final layer visualizes the machine’s decision process using colored circles and bars. Once it reaches a conclusion, the answer is shared on the touchscreen.

And that answer is sometimes wrong!

To Make it Fun, Don’t Make it Perfect

Ultimately, the neural network is about 85% accurate. Could we have expanded the data set to improve its recognition ability? Or added more nodes to create a more complex network? Sure. But that wasn’t the point. 

We kept things simple for two reasons. First, keeping things simple also created a more compelling and easy-to-understand visual. Too many connections would result in a messy visualization that’s impossible to trace or understand. 

Second, we wanted the neural network to make mistakes. We want people to push and prod the boundaries of the technology to understand where its limits are.

And that’s exactly what happens. 

Visitors don’t just draw one face and walk away; they draw a dozen. Friends gather around to watch the hologram in action. It’s quick, simple, and fun! And with every face, users begin to learn more about how the network works by witnessing how it thinks. Visit the MIT Museum on any given day, and you’ll likely find a group of friends bragging to each other about breaking the machine. There’s a sense of joy that comes from discovering its limitations.

Breaking the black box doesn’t ruin the experience; it makes it better. 

That’s exactly why MIT built its museum. To teach visitors about complex ideas through simple experiences. To give them hands-on experience with innovative technology.  Combining the mind (thought) and the hand (action) is at the core of the MIT experience. At Bluecadet, we’ve had so much fun helping them share that experience—that philosophy, really—with everyone.

Looking to increase engagement? Call us today and let’s talk about how AI can enrich your visitor experience.