MediaMonks Labs Explores the Future of AI-Powered Video Conferencing
MediaMonks Labs Explores the Future of AI-Powered Video Conferencing
How much time have you spent in video calls today? The answer is probably “too much,” and you may have a neck ache and a foggy, burnout-induced mind to prove it. While technology is invaluable for connecting people, too much time staring at a grid of faces can be exhausting for many—but there’s a better way.
To break away from the repetitive notion of video call after video call, the Labs team experimented with how machine learning can augment video conferencing for a better experience. The result is a game that challenges users to quickly find a random object and show it to their webcam. Using an object-detecting machine learning model to identify whether the chosen object is shown, the app awards a point to whoever finds it first. The frantic game not only offers an opportunity to get together and have fun at home, but also gets people moving.
Using a lightweight machine learning model that enables object detection locally within a web browser, the game is compatible with any video conferencing platform that allows screen sharing. Connecting is merely a matter of loading up the web page and sharing your screen—and then the fun begins.
The game’s development began as a way to rethink the potential of video conferencing, breaking away from the buttoned-up approach and offering a space for play and fostering relationships. “The basis of the idea came out of the COVID situation that led people to work and socialize remotely,” says Joe Mango, Creative Technologist on the MediaMonks Labs team. “We wondered: how can we use these tools that we have to build more fruitful and exciting interactions into mundane video calls?”
How Gamification Kickstarts Innovative Thinking
In challenging the notion of what a video call should look like, the team took inspiration from platforms that blur the boundaries between a standard call and a virtual space, like Online Town and Passtime—platforms that are almost more like games than just video calling. “Mainstream videoconferencing platforms are enterprise tools, but now we’re discovering people want more out of meeting each other,” says Geert Eichhorn, Innovation Director at MediaMonks. “We’re missing the spatial element, so you’re seeing gaming platforms competing with video calling platforms.”
This gamification element is an important piece of the web app’s development. It can be difficult to explain the potential of an emerging technology, as everyone approaches it with a different technical background and understanding. Upon coming up with the idea of using computer vision to augment video calls, the team found that gamifying the technology could help users easily wrap their heads around the feature. A humorous video that captures the fast-paced energy of the game aims to not only demonstrate the app’s capabilities, but also kickstart new ideas in viewers’ minds.
The frantic game challenges players to run around their homes to find random objects.
“Finding that human element that makes it fun is what made this project a success,” says Mango. “We realized we could make ourselves move around and laugh. The hardest thing with these projects is portraying how it relates to us as functional human beings in time and space, and we need to be pioneers on how it evolves.”
Anticipating the Future of Connection and Collaboration
Connecting digitally through collaboration across borders and in video calls isn’t new to the MediaMonks team, though working from home has prompted people across the company to rethink the role that video calls play throughout the day, both inside and outside of working hours. Sure, there might be a happy hour call to keep teammates connected, but Monks have also been known to 24/7 virtual hangouts that serve no specific purpose—where people might not even talk to one another for the most part—simply to replicate the feeling of sharing a communal space with other people.
Beginning out of a necessity to make business calls more engaging, Eichhorn envisions other uses of the computer vision and machine learning as well. “I think this is going to be another frontier,” he says. “We’ve just used off-the-shelf models for this project, but in other iterations we can train the model for different purposes. What if you wanted to train an AI to rate your work from home outfit before jumping in a call?” He compares the technology to the popularity of Snapchat filters that have made their way out of the mobile space and into online business meetings.
On that note, the technology (and the scope of Labs’ thinking) isn’t limited to just the context of video conferencing. It could also be used to power live experiential activations at home by allowing users to engage with digital objects overlaying their webcam’s image, a bit like interacting with AR. But by starting with the humble video call—that technology that we all know far too well by now—the gamified web app offers a fun and accessible way to imagine how we can work to improve the tools that we use each day to enable closer connection.
Video calls videoconferencing MM labs zoom google hangouts google meet livestream innovation zoom fatigue