With students holding graduation ceremonies in Minecraft and Travis Scott on tour performing via Fortnite, videogames have emerged and prevailed with ingenuity, with users finding new ways to communicate and socialize while at home. With a burgeoning streetwear market on Nintendo’s latest Animal Crossing release, brands are catching on to users’ craving to connect in virtual spaces.

This repurposing of platforms into digital venues might evoke the “metaverse,” or a persistent virtual world as seen in science fiction titles like “Ready Player One.” But as cultural hubs have been especially impacted by the COVID-19 pandemic, digital venues that replicate their real-world counterparts offer a great way to continue connecting with communities online. “Even though we’re seeing concerts and events move to the cloud and digital streams popping up everywhere,” says Geert Eichhorn, Director of Innovation at MediaMonks, “cultural venues, museums and other hotspots are commonly left behind.”

The Show Must Go On

Online virtual worlds certainly aren’t new, but few have captured mainstream adoption to become a killer app. Nor has there been a strong cultural drive to replicate in-person interactions virtually outside the context of gaming—until now.

“Right now, something we’re missing is the go-to platform that really owns that space,” says Eichhorn. “Second Life once had that promise, where brands would invest in virtual real estate on the platform. But given the global pandemic, it’s become more relevant to at least be prepared to have a virtual venue.” Linden Lab, the company that developed Second Life, has since built Sansar, a virtual reality entertainment platform recently sold to Wookey Project Corp. Facebook is working on a similar platform of their own called Horizon.

Over the years, we’ve seen museums in particular take steps toward this direction. Google Arts & Culture lets users explore world-renown museums using Streetview technology, for example, but the MediaMonks Labs team envisions fully realized 3D environments where multiple users can gather, mingle and engage in live content together. “With all of these venues currently being empty, now is the perfect time to shoot them through photogrammetry and build a custom digital space,” says Eichhorn.

Through technology like WebGL, MediaMonks has been able to design deeply engaging virtual spaces that require nothing but a web browser. With Victoria Cerveza, we built Mexico’s Biggest Offering by inviting people to honor the dead with a 3D altar. The result is a moving, collaborative tribute that users can contribute to or explore. Likewise, brands can embrace game platforms and accessible 3D technology to build culturally resonant spaces.

Free Your Venue from Physical Constraints

One of the greatest benefits of digital venues is that they allow brands and performers to continue engaging with communities at a time when consumers crave social connection and entertainment. But they’re also unbounded by physical constraints, opening up new opportunities to engage with content or personalize a space. “Consider entering a digitized museum, where you can swipe on a painting on the wall to view another by the same artist,” says Eichhorn. “You open up entirely new ways of interacting, and can even make it collaborative.”

Another idea is a personalized wall or wing filled with items from the collection, curated by data on viewers’ personal interests. “This level of interactivity and personalization applies to any experience you can do virtually,” says Eichhorn, noting that the technology is ideal for product demos, building creative customer experiences or conveying difficult concepts through visualization and virtual presentations.

There’s also no limit to the amount of people you can host within a virtual venue, enabling brands to extend their reach to wider audiences. In a world where social distancing regulations may be a part of our new normal, this could be important than ever. While platform constraints might require overflowing attendees into multiple servers or “instances,” there’s no need to shut the door on anyone. “With digital venues, you can give everyone a front-row seat if you wanted to,” says Eichhorn.

Considerations for Building Your Digital Venue

Above, we’ve discussed translating a physical space into a digital one. But brands can build entirely imagined digital venues as well. Whether you aim to bring a fictional branded space to life or want to meticulously render existing products to demo virtually, a production partner who’s skilled in VFX and digital production can provide guidance on building a space that’s not only functional, but drives emotional impact—a critical factor that’s often missing when programming is translated from a physical space to just a livestreaming platform.

For example, it’s important that you don’t just focus on visual fidelity when digitizing an existing space. Aim to replicate other senses however possible in order to add a greater sense of atmosphere. Eichhorn highlights the 360-degree work we did for Bancolombia: School of Sustainability, which not only makes a remote island community explorable, but brings it to life through the use of sound.

“We went in there to do 360 photography, but we also caught a bunch of ambient audio because the island is so densely populated, which made it feel truly alive,” says Eichhorn. “If you’re shooting in an empty place that’s harder, but brands can still make it feel like people are there with the user.”

In addition to creative considerations, brands must also take a moment to consider which platform suits their goals—or their audience—best for hosting a digital venue. Game platforms may be appealing to younger audiences, for example, while platforms like Mozilla Hubs have a low barrier of entry by being available directly within a web browser. No matter the purpose for your digital venue, we can create scalable, proprietary platforms to host virtual events—and even include social features and monetization of content.

By offering digital spaces for relaxation, play and continued programming, brands can tap into a behavior that consumers have already gotten used to in response to staying at home: meeting in virtual space. No longer constrained by physical limitations, these digital venues enable new ways of interacting with patrons and can continue extending experiences to distant audiences in the future. But most importantly, they enable brands to continue serving an important aspect of consumers’ daily lives that has become difficult to hold onto: enjoying cultural touchstones together.

Rethink real-world events for digital.

Engrossed in virtual reality, you’re surrounded by digital, fantastic objects, each begging for you to reach out and touch. But until recently, most interaction in mainstream VR headsets has still been limited to using a controller. For some experiences, the controller presents a disconnect between what people feel in their hands versus see on the screen—at least until recently.

Last month, the Oculus released its Hand Tracking SDK for the Oculus Quest, allowing people to use their hands to navigate through menus and applications supporting the new SDK. While the update isn’t meant to replace controllers outright, it enhances users’ sense of presence within the virtual space by blurring the barriers between real and virtual even further, presenting new creative opportunities for brands that are eager to offer assistive content in the emergent medium. “Tangibility in digital has always been equated to a click of a mouse or key, but now it’s becoming even more of a physical thing, more like a real experience,” says Geert Eichhorn, Innovation Director at MediaMonks.

This illusion of reality is intriguing for Seth van het Kaar, Unity Monk at MediaMonks. “One thing VR has shown through experience and research is that our eyes override our other senses,” he says. “So, if I appear to be putting my hand in a bucket of cold water in VR, I’ll get the placebo effect of it feeling cold. Through creativity, you can use that to your advantage.”

Exploring the creative opportunities presented by the SDK, van het Kaar served as developer on a team of Monks experimenting with hand tracking to develop a working prototype that could take best advantage of the new interface. Here’s what the team learned in the process.

Find Opportunities to Get Hands-On

“Similar to how the development of voice as an interface has prompted brands to emulate human conversation as naturally as possible, we need to make these experiences feel as intuitive as possible, as you’re using your real hands,” says Eichhorn. As part of MediaMonks Labs, our research and innovation team, he’s focused not on using the latest tech for the sake of it, but rather finding the real-world application and value that it has for end-users.

Trying to identify what type experience would best benefit from this new input, the team wondered: what activities are very dexterous and require careful use of one’s hands? Shaving made sense: “It’s something that’s difficult for young adults and teens who are just learning to use these devices,” says Eichhorn. “And a lot of people still get things wrong, like going against the grain.” It’s also an intriguing use case in that shaving requires an element of precision, putting the usability of hand tracking to the test.

By practicing grooming in VR using one’s own hands, users would be able to try out different tools and techniques without worrying about messing up their own hair. So, the team took our bald monk mascot and blessed him a head of hair, inviting Oculus Quest users to give him a shave and a trim in an experience inspired by the Play-Doh “Crazy Cuts” line of toys.

Start with Something Familiar

Interacting with one’s hands is incredibly intuitive; it’s one of the earliest ways that we engage with the world as infants. But that doesn’t mean any hand-tracking experience is inherently easier to use or design; experimenting with any new mode of interaction requires one to break free of any preconceived notions about design. In the case of hand tracking, how does one organize a series of options within an experience without the use of physical buttons (and in this case, no haptic feedback)?

To rise above the challenge, the team used common hand gestures as a starting point—for example, those used in rock/paper/scissors—to serve as an intuitive metaphor for interaction.  “The Oculus can track the difference between fingertips, so if I mimic scissors with them, that’s a funny interaction,” says van het Kaar. “In the app, you can select the scissors and now you’re like Edward Scissorhands,” a fictional film character whose hands made of scissors give him wild success as a hairstylist.

Move Beyond Limitations and Creative Constraint

In its experiments with the SDK, the team settled on a couple of learnings that could apply to subsequent hand-activated Oculus Quest experiences. First, there’s moving past the challenge felt in any VR environment: locomotion, or the relationship and (de)synchronization between one’s bodily movements and those of their virtual avatar.

Without haptic feedback, what should happen when the user’s hand comes in contact with a virtual object: should it move through the object, or should the object block their movement much like it would in reality? While the latter option might make sense on paper, the fact that users could still move their physical hand while the virtual one stays stationary could result in confusion. The team moved beyond the challenge by letting users push virtual objects freely—for example, the monk model that they shave—which snap back into place once released (which sounds like a fun interaction in its own right).

The way that Hand Tracking SDK detects hands also presented a challenge: it seeks out the shape of a hand against a background, so it loses tracking once two of them overlap. “You can’t place a menu on the palm of your hand and tap an option on it, or interact with a virtual object on your wrist, for example,” says van het Kaar. To work around this challenge, a menu floats beside the user’s hand. While this doesn’t allow for haptic feedback by selecting options against one’s own body, this setup mitigates the risk of losing the tracking by having hands overlap.

Taking the time to experiment and apply these learnings allow us to develop increasingly realistic experiences in extended reality. From playing with hand tracking in VR to demonstrating how occlusion transforms experiences in AR, our team of makers are devoted to continually experimenting with new technologies, finding their most relevant use cases and establishing best practices for brands and our partners. As barriers continue to break down between the physical and virtual, it will be exciting to see what kinds of wholly new digital experiences emerge.

The Mobile World Congress took to Los Angeles this week, gathering together the wireless industry: network operators, software companies, manufacturers and creative partners. The focus on this year’s event was intelligent connectivity, and how 5G is set to bridge together several innovations (like big data, AI, the internet of things and extended reality) to reinvent the way we interact with content and each other, both in our professional and daily lives.

In his keynote kicking off the event, GSMA Director General Mats Granryd identified several real-world impacts that 5G will offer. Mental health practitioners could provide at-home therapy to lonely patients via hologram, for example; colleagues could better collaborate with one another in real time from across the world, and students could “literally carry your classroom in your pocket” with experiences make a greater impact than a simple video recording of a lesson.

It’s the Year 5G Finally Gets Real

At a gathering of so many innovators and mobile operators, you get the sense that anticipation for “what’s next” is high. Technologists have waited years for 5G to grow out of its status as a buzzword and into an actual offering. With its rollout to select cities in the US, the promise of the ultra-fast connection is almost upon us, and a sense of excitement permeated the conference. In conversation with Meredith Atwell Baker (President and CEO, CTIA), Ken Meyers (President and CEO, US Cellular) contrasted this attitude with the jump from 3G to 4G. “We didn’t sit back and think, ‘Oh, look at the app-based economy right in front of us,’” he said.

But that’s what’s happening now. On the panel “New Marketing Strategies: How to Make Money with XR,” RYOT Head of Content Nigel Tierney mentioned how even with 5G on the horizon, there are still limitations to solve: “We’re at the crux of unlocking possibilities.” Silkie Meixner, Partner, Digital Business Strategy at IBM, likewise mentioned how the firm is working now to help clients prepare for a future that’s ripe with opportunity and is set to change the way they work. “There are already things we have to do with our clients to think 5 years out,” she said. “You have to take 5G as a given.”

Buying in on Big Bandwidth

So, what does the 5-year, 5G plan look like? The simplest way to envision a 5G-infused future is to consider the significant boost in bandwidth it will provide: it can reach speeds of up to 100x faster than 4G, which had previously made significant impact on services like streaming music and video years ago. And what 4G connectivity has done for video, 5G could do for emerging media, including cloud-based gaming (like Google’s upcoming Stadia gaming service) or streamable AR and VR.

In his keynote presentation, Viacom CEO Bob Bakish discussed how the multinational entertainment conglomerate is looking forward to a near-future of premium video content, enhanced with the power of 5G and integrated with related media and platforms. This would usher in an “era of advanced video experiences that will truly allow mobile to distinguish itself as an entertainment medium.” It would also encourage brands and content creators to consider the many contexts in which audiences will connect to their content: on a smartphone, in a driverless car or somewhere else.

Bakish mentioned how developing such content through new partnerships would help network operators differentiate themselves by leveraging their partners’ IP. We saw the strategy in action at this year’s Comic Con, where we helped AT&T launch a VR experience that let Batman fans fly through Gotham City. The experience’s presence at the conference instilled trust in AT&T’s ability to output the kind of content that audiences desire after its acquisition of Time Warner and DC Comics IP.

Extending Extended Reality Even Further

While the Batman experience was site-specific, 5G offers opportunity to enhance and scale up such experiences for mass audiences. One of the biggest challenges affecting AR and VR right now is that it’s not easily streamable; users must discover and download applications for fully-featured experiences, which is partly why the much more limited (yet accessible) camera filter has risen as the most popular and ubiquitous use of the technology.

But 5G can do away with those constraints. “More bandwidth means we can be more ambitious and artistic with the content we create,” says Olivier Koelemij, Managing Director of MediaMonks LA, who sat on the same panel. “A better, more immersive story means our strategies to amplify it will become more ambitious in lockstep.”

This means there’s opportunity for brands to relate with audiences through more sophisticated, shareable digital experiences. Tierney attributes failure to poor storytelling and lack of meaningfulness, citing a need for brands to integrate personalization and data into the creative process and delivery—basically, they need to be more purposeful in their ideation and delivery to provide resonant interactive experiences.

“We don’t suggest a technological approach because it’s the hot trend,” says Koelemij. “Our content and technology must be fit for format, purpose and consumer.” He suggests viewing any creative problem through a pragmatic lens. “You should ask questions like: is extended reality helping us deliver a stronger message here? How can we integrate other digital elements to do this?” The goal is to home in on the right approach for your business goals through data and KPIs, ultimately delivering an experience that resonates with consumers.

Meixner described how IBM employed such a strategy to develop a VR-enabled training experience. The B2B solution not only makes training faster and scalable by teaching trainees skills that they’d otherwise gain in a classroom—it also collects data through interactions like motion analysis, which could be used to optimize the tool or develop new ones. The strategy shows how innovations can be developed and optimized through practical, real-world data that empowers and educates.

It’s clear from this year’s conference that 5G isn’t just about connecting people to friends or family via a wireless handset. It’s about truly integrating all of the devices and touch points we interact with each day, enabling truly transformable and new interactions. As the technology begins to roll out, brands must be prepared to adopt it with a sense of purpose to offer audiences meaningful, impactful and differentiated experiences.

This week, Fast Company launched its annual Innovation Festival, featuring Fast Track sessions that take attendees behind the scenes and into the homes of some of the most innovative companies in New York City. Billed as “Fast Company’s unique take on the field trip,” Fast Track sessions engage brands and creatives through hands-on talks and experiences hosted by participating companies, including MediaMonks.

Our New York office opened the doors to the dojo, inviting brands into our home to discuss all things extended reality. In a panel session devoted to augmented reality and its application to music and entertainment, our Monks dove deep into the design and development process of Pharos AR—a mobile AR experience made in collaboration with Childish Gambino, Wieden+Kennedy, Unity and Google. Taking users on a cosmic journey set against an exclusive track from Childish Gambino, the app is notable for being the first multiplayer music video.

With a panel including Snider-Held (Creative Technologist, MediaMonks), Thomas Prevot (Senior Producer, MediaMonks) and Tony Parisi (Head of VR/AR Brand Solutions, Unity), the session served as a casual fireside chat. The conversation kicked off by establishing the state of VR and AR, often characterized by the conflicting feelings that VR is dead and that the clear use case for AR hasn’t yet been found. But both technologies are well established, each excelling in achieving different goals within different environments.

Showcasing our Batman experience as a strong example of the immersive powers of VR, Snider-Held noted that “These experiences are still very installation-based,” and that AR’s distribution through mobile offers the potential for greater reach with a simpler experience. In explaining the process of developing Pharos AR in particular, the group explored key considerations for challenges to consider when developing an extended reality experience.

AR Can Feel Real Without Being Photoreal

Constraint prompts creativity—an adage that applies just as well to AR as any other medium for art making. Because mobile AR experiences are designed for use across a variety of devices, they must be relatively lightweight to provide a smooth experience to the widest share of users. Failure to keep technical constraints at top of mind can instead result in a lagging, stuttering experience that breaks immersion.

While this is true for any digital experience, it’s especially true for AR, a medium which Parisi says aims to “intelligently interact with the real world.” This expectation to seamlessly integrate with the surrounding environment can make stuttering graphics stick out like a sore thumb. “You want to keep the frame rate above 30 frames per second,” says Snider-Held, “because the user will compare the motion on the screen with the action happening around them in reality.”

The trio took this challenge as an opportunity to discuss the highly stylized look achieved with Pharos AR. While a photorealistic graphics might be impractical for a mobile device to realistically render in real time, a stylized look presents the opportunity to differentiate your experience through a strategic choice in aesthetic; for Pharos AR, the team took visual inspiration from Childish Gambino’s laser-punctuated stage shows, ensuring the app’s look and feel naturally integrated with the rest of the artist’s visual brand.

“Stylistically, we were trying to remain within the constraints of mobile processing in a visually appealing way,” said Snider-Held. An example of this is the use of particle effects, in which sparkles of light coalesce into a ghostly image of Childish Gambino as he dances to the music, animated via motion capture. “This is the best example on why you don’t need to do photorealism,” Parisi said. “We were able to capture the essence of Donald, because it’s his dance.”

Carefully Plan the Narrative Environment

Extended reality experiences are interactive by nature, meaning they rely on a different approach than how you would plot out and plan more linear experiences. There’s a careful balancing act between giving users the reigns to explore on their own versus stringing them along a narrative thread. MediaMonks Founder and COO Wesley ter Haar notes that “Narrative UI is key for onboarding and guiding the user in an AR experience,” making it incredibly important that you plan out users’ interactions and use environmental cues to shape a narrative.

While Pharos AR begins and ends through open-ended user interaction, it still follows a clear narrative through the virtual performance of Childish Gambino’s single, “Alogrhythm.” In exploring the primary path through the experience, the team began planning it in storyboard form, much like you would make for a film. “This process not only serves as visual research, but also in briefing the animation team and envisioning how actions will play out in the user’s environment,” said Snider-Held.

That makes sense—but what sets storyboarding for AR or VR different than other forms of digital storytelling? Because users could play with the app in their homes, the team had to plan for other variables how large the virtual scene should be, and what actions would be possible for multiple users to make within a smaller environment—like the cramped living room of a New York apartment, to offer an example that the Fast Track attendees could relate to. The challenge demonstrates how important it is to map out the virtual scene for different scenarios and users.

An interesting insight uncovered in the panel was that the team didn’t just rely on visual methods of planning like maps and storyboards. Due to the nature of the background music building up as users explore the space around them, the team also developed a musical timeline that maps up how different interactions trigger the layering of the music. The step showcases how sensorial, environmental cues can shape the action within an immersive, extended reality experience.

Whether developing for VR or AR, extended reality experiences require developers to rethink the creative approach beyond the standard linear story. From considerations in setting, technical constraints and variations in number of users across platforms, extended reality development relies on a comprehensive understanding of the building blocks that make up a total user experience. Snider-Held capped off the session with an ambition on what MediaMonks aims to achieve with brands through such experiences: “We strive to further the use of the technology from impossible to probable, and experiment in how to further that, too.”

Granted, it’s been a while since I’ve been at the student’s end of a classroom, but I’d like to think I haven’t totally lost touch with my childhood education. And as much as I enjoyed being in school, my fondest memories—prior to discovering the internet—are probably those that were most hands-on.

Field trips and practical lessons gave me the chance to learn on my own terms. Rather than learning through the filter of others, these outings allowed me to create my own educational narrative and learn through first-hand experience—escaping the usual set up where you sat staring at a teacher as they talked at you from the front of the room.

However, since my earlier expeditions to the local science museum or treatment plant—yes, my school was very progressive—educational technology (edtech) has changed the game considerably. Smartboards (digital whiteboards), laptops, and smartphones have made their way into the classroom and the curriculum. And as with anything concerning children, this influx of technology in the classroom adds fuel to the fire of the helpful vs. harmful debate.

We have to deal with the fact that progress is inevitable. My parents for one, missed out on most of the digital revolution, leaving them oblivious to a lot of its benefits. Who would have expected that with just a smartphone and a selfie-stick, you can discover parts of life on earth that are otherwise invisible or out of reach? That’s the innovation that Google Expeditions, an educational tool launched in 2017, offers by showcasing the promising ways in which AR and VR can turn your average lesson into an extraordinary expedition.

While we might not be able to avoid progress, we can influence it by deciding how to expose future generations to innovative tech. What better place to start than by using innovative tech to take children to places beyond their wildest imagination—creating experiences that even field trips can’t match?

Back to the Chalkboard

I’ve never met anyone who learned to play football just by reading about Cruijff’s biggest feats alone or could ride a bicycle after watching Peter Sagan pull-off a winning sprint. If we want to develop certain skills and learn valuable lessons, we have to live them. And even though offline teaching increasingly revolves around exploration, there’s still a gap between many areas of applied learning and everyday practice.

Why not close the gap by using technological advancements to our advantage? By giving children access to experiences that are quite literally out of this world, we can further encourage the development of real-world skills—without having to leave the classroom. 

Others are taking note of this potential; investments dollars have risen in edtech over the years, with over $1.45 billion raised by US edtech startups in 2018—beating the $1.2 billion raised the year before. But that money is also going into fewer and fewer companies, demonstrating a need for edtech tools and startups to stand out with unique experiences and demonstrable educational results.

As this increasingly competitive environment matures, the winning edtech startups are those that provide differentiated experiences. They won’t use tech for tech’s sake, but will find the best medium for the message and iterate upon it. Sitting for hours on end scrolling through our feeds is not something you want to actively encourage, but exploring a virtual visualization of a DNA string is.

Designing these experiences requires educators to build an appetite to try new things and innovate—essentially, they must become students themselves, seeking out ways to innovate. A publisher can translate their book into a CD or internet portal, which might offer some interactivity and exploration, but that does little to make the lesson significantly more engaging or meaningful to a student. Instead, edtech requires an entirely new mentality around learning.

The Next Frontier  

It’s not just earthly life that’s suddenly at our fingertips thanks to new technical interfaces. VR offers the first real means of making “space travel” possible for everyone. The SPACEBUZZ project is something particularly close to my heart. TLDR: a real-life rocket pulls up to schools, virtually launching students into orbit.

While working on SPACEBUZZ, I got to see first-hand how an interactive VR experience can leave a lasting impression on developing minds and encourage curiosity. Guided by real-life astronaut André Kuipers (who helped develop a fact-based script and serves as Mission Commander in the experience), kids get to gear up and produce in a 15-minute space flight.

What really makes a difference is giving children the hands-on experience to learn. While children who watched just the VR flyover film were impressed by the view of earth from space, the students who experienced the onboarding mission first got into the mindset to better investigate and understand what they were seeing from the stellar view—like the visible effects of deforestation, air pollution and more. Similarly, the Lockheed Martin initiative, Field Trip to Mars, shows just how much influence a positive application of technology can have on the appetite for learning, bolstering a positive perception of digital experiences.

By embedding practical technology in the everyday lives of the next generation, in the right setting, we’ll not only see a continuation of positive technological experiences but also provide access to a broader educational experience. In addition to giving children a new perspective on the world around them, these digital experiences could inspire deep engagement with their passions across a variety of subjects, helping cultivate the next generation of scientists, teachers, astronauts, manufacturers and more. We could tailor parts of these experiences to speak to individual needs, because who knew that according to one study girls learn better from a virtual teacher while boys prefer tutoring from a drone?

The above gets at the heart of what makes edtech such a powerful tool: teach students in a scalable way that’s personalized to their individual learning styles. In a Medium post that serves as a primer to the different styles of personalized learning, the Office of Educational Technology within the US Department of Education defines personalized learning as “instruction in which the pace of learning and the instructional approach are optimized for the needs of each learner.” The key thing is that “learning activities are made available that are meaningful and relevant to learners, driven by their interests and often self-initiated,” and edtech offers an excellent opportunity to prompt that sense of exploration in students.

With this potential for individual enrichment, edtech doesn’t have to be limited to education throughout the formative years. Imagine what we could do if we applied the same theory throughout the entire educational infrastructure? In supporting lifelong learning, edtech could play a key role in training across disciplines, including fields in which the stakes are high, like in the medical industry, combat training or of course: space travel.

From digital experiences that inspire to the possibility of personalized courses and expert guidance based on personal data, edtech can set students up for a lifetime of learning, helping them discover and apply their passions to benefit society as a whole.

I might not be in formal education anymore, but I’m definitely still learning—and look forward to continue paying that experience forward to help tech users of any age better explore and relate to the world around them.

First, you feel a thrilling sense of weightlessness as you dive through the city skyline, watching skyscrapers pass you by. But the exhilarating skydive takes a harrowing turn when you spot your target: the Scarecrow, who’s wreaking havoc throughout Gotham City by spreading his trademark toxic fear gas.

As you cut through a gas cloud, your vision becomes clouded: villains from the Batman universe begin to surround you as buildings come crashing down. As you glide through the increasingly abstract and crumbling city, can you catch the villain in time?

This isn’t your typical skydiving experience. It’s just one part of the Batman Experience, powered by AT&T at Comic-Con, celebrating the character’s 80th anniversary and induction into the Comic-Con Character Hall of Fame. The activation made such an impression that Wired said it “soars—while others fall flat,” while the New York Times called Batman and AT&T “a new dynamic duo.” But how exactly does such a heroic tale come to be?

Every Kid’s Dream: Be Batman

In a recent SoDA Report On Trends in AR, VR and Mixed Reality, MediaMonks Creative Technologist Samuel Snider-Held writes: “[AR and VR are] no longer the shiny, new thing it once was, and brands can’t blame a subpar experience on experimentation, either.” Rather than chase the appearance of innovation, he says, brands must closely consider “how a given touchpoint impacts the broader customer experience” and design the offering around that. The VR segment of the Batman Experience achieves this by pairing the virtual scenario—gliding through the city to catch Scarecrow—with the weightlessness made possible by a skydiving wind tunnel.

We all know about Batman’s utility belt full of cutting-edge tools and equipment. This VR scenario lets everyday people experience what it’s really like to try out the kind of tech Bruce Wayne—Batman’s alter-ego—would use. But the immersion begins before participants put on the custom headset shaped like Batman’s iconic cowl, with an instructional safety video delivered by Lucius Fox, who supplies Batman with his equipment.

Contrasting the Batman Experience with other AR and VR activations that fell short of pushing the medium forward, Wired noted that AT&T’s stood out among others at Comic-Con this year: “At all of 77 seconds, it’s not long … But it’s also utterly unprecedented.” It didn’t take much time to make a powerful impact. The cinematic narrative thread throughout the experience, from onboarding to leaving, goes a long way in ensuring that participants are fully engrossed in the world of Gotham City.

The video briefing, for example, turns a boring safety regulation—required for these sorts of events—into an opportunity for storytelling. “We really wanted to immerse participants into the role of Batman in a full narrative sequence,” says Eric Shamlin, SVP Growth at MediaMonks, who collaborated with AT&T to bring the project to life. “Part of this idea was that Lucius would do the onboarding video to showcase a new adaptation of the bat suit, training them on how it’s used.”

Large-Scale Activations Still Benefit Smaller, Niche Audiences

The tech-infused experience showcases the strength of the telecom company’s network that powers many of the things that Comic-Con’s audience cares about: things like streaming movies or TV shows, online gaming experiences and more. By paying homage to 80 years of Batman in the opening of the Comic-Con Museum, AT&T pays respect to the franchises and characters that the corporation recently inherited in its acquisition of Time Warner.

The exhibit also gives attendees an idea of the partnership’s increased creative and production muscle. “The combination of the two companies was intended to create something new in the media industry,” writes the New York Times covering the exhibit. “A powerhouse that could reach millions of people through its vast distribution system of mobile devices and satellite networks, while also creating the content that will fill their screens.” Through an exhilarating cinematic activation, fans get a taste of the ways AT&T might bring its properties to life on and off the screen in the near future.

Following insights from this year’s Cannes International Festival of Creativity, Accenture Interactive highlighted the need for brands to “think small to launch big, going after enthusiasts in need. It’s a need for niche rather than mass.”

A dependable strategy, even with large-scale activations like this one, is to start small by first targeting niche audiences by delivering the experiences they crave. While superheroes aren’t necessarily a niche interest these days—they continue to dominate in box office sales, and Batman himself is one of DC’s most popular characters—it’s worth noting that San Diego Comic-Con’s audience makes up some of the most passionate fans of comic culture.

And while the superhero-loving audience may be growing, it’s also a skeptical one, highlighting the importance in delivering exceptional creative to those ends. “AT&T might be viewed by some as a heavy-handed corporate voice,” says Shamlin, “but they’re successfully navigating how to leverage the very significant pop culture icons they’ve come to control to make themselves more approachable to a very discerning audience.”

Ensuring an Experience for Everyone

The team knew that not everyone attending the museum would have the chance to try out the VR experience that stole the show. “We didn’t want those who missed a chance to try the skydiving experience to feel they missed out, so we designed the entirety of the activation to be fun and shareable,” said Shamlin. The multi-layered activation made full use of the Comic-Con Museum, bringing attendees the chance to engage with the Batman universe in a variety of ways.

This included a Bat Cave, built by MediaMonks, where attendees could play through an archive of Batman video games. We also developed a training room filled with punching bags featuring portraits of villains from the franchise. As you punch the bags, light projections dynamically fill the walls with comic panels and onomatopoeia (“Pow! Wham!”) that let you feel like you’re in a comic book—or the campy Batman TV series from the 60s. Throughout the museum, attendees could pause to snap a selfie with a variety of props from Batman movies, including costumes and the iconic Batmobile.

Pulling all of this off was a heroic effort in itself, requiring the alignment of several parties: AT&T and its agency The Collective; Comic-Con, including the group managing the Comic-Con Character Hall of Fame; and the city of San Diego itself, who issued the required permits for hosting the massive stage for the skydiving experience. “Our activation was just one part of a larger one within the museum,” said Shamlin. “There was an incredible amount of coordination and back-and-forth communication.”

From aligning a diversity of experiences to getting several partners (and fans) on the same page, an exhibit on the level of the Batman Experience is a huge undertaking. But together, the multiple layers of the activation culminate in a rich, accessible exploration of Batman, his history and his cultural impact over the decades—a recognition that the character doesn’t need, but certainly deserves.

With omnipresent technology on the horizon, are we headed toward a cyberpunk dystopia?

That was the big question MediaMonks posed to the audience at its Fast Co. Fast Track session, offering attendees food for thought to mull over with drinks. Rejecting a future in which pop-ups and banner ads vie for our attention literally everywhere we look, speakers Eric Wagliardo and Jouke Vuurmans offered a more optimistic view. By using emerging technology like VR, AR and artificial intelligence to delight audiences, Wagliardo told the audience, “we can use creativity unconstrained by the past and the laws of physics.”

The theme of the talk anticipates a near-future in which wearables or ubiquitous AR will revolutionize the way we interact with media and technology, just like how the iPhone ushered in the era of smartphones. As these technologies become more consumer-friendly, brands might be biting their nails wondering if they’re ready for the new digital landscape. Are you?

The focus of your VR or AR experience doesn’t have to be grand to be impressive.

One of the more compelling details from the talk was about a humble plank of wood used in the Jack Ryan Experience, a 60,000 square-foot activation that pulled together some of the most cutting-edge VR technology to provide a 4D, immersive experience. Through a VR headset, the plank transformed into a startling chasm for participants to cross in virtual reality. That such a basic material in reality could become an intimidating environment to traverse demonstrates the power to create compelling experiences from just a simple object. With so many elements and gorgeous tech to compete with, who would expect a plank of wood to steal the show?

VR can be tricky, though: without a proper “hero device” like the iPod or iPhone to put the tech in almost everyone’s pockets, VR experiences must often be tethered to a specific environment for participants to don a headset and enter your world. This can be a big investment in terms of reserving a space as well as providing a rig to host the experience, though it’s well worth it if you’re hosted within a venue that allows you to reach your audience in a meaningful way. For some brands, a simpler solution might be to create an AR experience that participants may enjoy via nearly any mobile device.

Another benefit to AR is that it can enable a charming connection with objects or pre-existing campaigns that wasn’t possible before. One of the demos featured at the Fast Track event invited attendees to scan an ad for Kiwi shoe polish featuring a Van Gogh self-portrait, which prompted the artist to leap from the poster, ready to try on several pairs of shoes with the user.

The AR app was a remix of an award-winning Ogilvy campaign that “completed” various famous portraits by adding views of the subjects’ shoes. Partnering with Ogilvy and Google, MediaMonks was able to bring the Van Gogh portrait to life by stepping into the real world and responding to different shoe styles with animations and props. The experience of helping the famous artist try on shoes is humorous and helps the user relate to an esoteric character they certainly recognize, but may not have previously understood.

AR and VR are more consumer-friendly each day, offering several ways for you to reach them with ease.

 A large-scale project with emerging tech can be intimidating. While it can be difficult to identify where to begin planning bigger projects, there are several levels of engagement to design for in VR and AR, allowing brands who want to dive into emerging tech to start small and provide increasingly immersive experiences from there. One way to provide an engaging experience is to focus on one simple, primary mode of interaction. For example, the Kiwi case demoed at the Fast Track event required only a poster and a mobile device to render the character on the screen, along with some options for interacting with that character. This simple template could be useful for brands new to the emerging tech space, who also want to provide engaging, emotional experiences.

Seeing is believing.

And just like how the app brings the print ad featuring Van Gogh to life, you can use your app to remix existing assets from current or past campaigns, thereby boosting their ROI—not bad for a budget-strapped brand who wants to dive in on AR.

If a business has a little bit of extra room in their budget, they can elevate the same experience detailed above by staging it in an appealing way. While the primary mode of interaction might be to bring a character, object or other experience into the real world via AR, offering a set for participants to play with the experience can further immerse them. In the case of the Kiwi app, MediaMonks constructed a tiny room at Cannes Lions International Festival of Creativity decorated to look like a city street you might find in a Van Gogh portrait, allowing participants to step into the artist’s world (without the need for a VR rig and other expensive tech).

If one theme is clear across the Innovation Festival, it’s that the quickening pace of new media and platforms are challenging brands to think outside the box in providing new, experiences to enchant and inform their audiences. We were happy to allow guests into the hallowed chambers of the MediaMonks monastery to impart a secret or two about our process. But can the students surpass the master? We look forward to seeing what you create!

If you thought that mind control was only real in Marvel’s cinematic universe then think again.

Vodafone announced the launch of a 4G network on the moon in 2019—and since it’s now possible to swim with sharks in VR using 4G, why wouldn’t we try to raise the bar even higher to transmit brainwaves via 4G?

Together with Vodafone, ACHTUNG! mcgarrybowen, and Made.For.Digital. we made it possible for anyone to feel like Charles Xavier and control a race car in virtual reality (VR) — and real life — with their brain.

Stage One

The first stage of the project, dubbed Formula Brain (the pun works a tad better in my native Dutch tongue), launched earlier this year in twelve Vodafone stores across the Netherlands.

Once store visitors took place in the driver’s seat, they were fitted with Emotiv EEG headsets, which allows for the brainwave frequencies to be registered and transmitted. The headset calibrates four types of brainwaves ranging from low to high frequency — Theta, Alpha, Beta and Gamma.

The frequency measured by the EEG headset is then filtered into code that was used to control the car. In short; higher focus means a higher frequency and so higher acceleration. To complete the life-like feel of the experience, we used Gear VR headsets that immersed racers in a circuit based on the Red Bull Ring.

Stage Two

Over the course of three weekends, participants competed for a coveted ticket to the second stage: a race event for the five best racers at a local circuit. Using the same EEG headsets for measuring brainwaves, the real life race event offered a more realistic driver’s experience — complete with Vodafone suits, live telemetry data reflecting the car’s performance, and a direct camera feed from inside the car for the racers to focus on.

Inside a custom-built dome, placed in the circuit’s center, the racers claimed the driver’s seat once again — this time controlling a physical race car on the track outside. Vodafone’s 4G network was the connecting factor in this stage of the game, making sure the racers brainwaves were transmitted to the car on the track — all in real time. In the end, one racer was crowned the king of Formula Brain and won a ticket to the 2018 Grand Prix of Austria at the Red Bull Ring.

Brainwaves + 4G Network = Dream Team

Keeping in mind that Formula 1 drivers go through years of extensive training to maintain focus for seventy laps at soaring speeds, it’s even more exciting that we allowed regular people to have the same experience with barely any training — all using the power of their brain.

Adding our experience with VR and real time telemetry data handling to the equation — think of projects like Red Bull: Air Race Live VR, the Old Spice: S.Q.U.I.D. and more recently Nike Game of Go — promises more exciting possibilities for the future.

Maybe down the road we’ll see what else these technologies holds for creating some high-octane brand experiences — on this planet and beyond.