Engrossed in virtual reality, you’re surrounded by digital, fantastic objects, each begging for you to reach out and touch. But until recently, most interaction in mainstream VR headsets has still been limited to using a controller. For some experiences, the controller presents a disconnect between what people feel in their hands versus see on the screen—at least until recently.

Last month, the Oculus released its Hand Tracking SDK for the Oculus Quest, allowing people to use their hands to navigate through menus and applications supporting the new SDK. While the update isn’t meant to replace controllers outright, it enhances users’ sense of presence within the virtual space by blurring the barriers between real and virtual even further, presenting new creative opportunities for brands that are eager to offer assistive content in the emergent medium. “Tangibility in digital has always been equated to a click of a mouse or key, but now it’s becoming even more of a physical thing, more like a real experience,” says Geert Eichhorn, Innovation Director at MediaMonks.

This illusion of reality is intriguing for Seth van het Kaar, Unity Monk at MediaMonks. “One thing VR has shown through experience and research is that our eyes override our other senses,” he says. “So, if I appear to be putting my hand in a bucket of cold water in VR, I’ll get the placebo effect of it feeling cold. Through creativity, you can use that to your advantage.”

Exploring the creative opportunities presented by the SDK, van het Kaar served as developer on a team of Monks experimenting with hand tracking to develop a working prototype that could take best advantage of the new interface. Here’s what the team learned in the process.

Find Opportunities to Get Hands-On

“Similar to how the development of voice as an interface has prompted brands to emulate human conversation as naturally as possible, we need to make these experiences feel as intuitive as possible, as you’re using your real hands,” says Eichhorn. As part of MediaMonks Labs, our research and innovation team, he’s focused not on using the latest tech for the sake of it, but rather finding the real-world application and value that it has for end-users.

Trying to identify what type experience would best benefit from this new input, the team wondered: what activities are very dexterous and require careful use of one’s hands? Shaving made sense: “It’s something that’s difficult for young adults and teens who are just learning to use these devices,” says Eichhorn. “And a lot of people still get things wrong, like going against the grain.” It’s also an intriguing use case in that shaving requires an element of precision, putting the usability of hand tracking to the test.

By practicing grooming in VR using one’s own hands, users would be able to try out different tools and techniques without worrying about messing up their own hair. So, the team took our bald monk mascot and blessed him a head of hair, inviting Oculus Quest users to give him a shave and a trim in an experience inspired by the Play-Doh “Crazy Cuts” line of toys.

Start with Something Familiar

Interacting with one’s hands is incredibly intuitive; it’s one of the earliest ways that we engage with the world as infants. But that doesn’t mean any hand-tracking experience is inherently easier to use or design; experimenting with any new mode of interaction requires one to break free of any preconceived notions about design. In the case of hand tracking, how does one organize a series of options within an experience without the use of physical buttons (and in this case, no haptic feedback)?

To rise above the challenge, the team used common hand gestures as a starting point—for example, those used in rock/paper/scissors—to serve as an intuitive metaphor for interaction.  “The Oculus can track the difference between fingertips, so if I mimic scissors with them, that’s a funny interaction,” says van het Kaar. “In the app, you can select the scissors and now you’re like Edward Scissorhands,” a fictional film character whose hands made of scissors give him wild success as a hairstylist.

Move Beyond Limitations and Creative Constraint

In its experiments with the SDK, the team settled on a couple of learnings that could apply to subsequent hand-activated Oculus Quest experiences. First, there’s moving past the challenge felt in any VR environment: locomotion, or the relationship and (de)synchronization between one’s bodily movements and those of their virtual avatar.

Without haptic feedback, what should happen when the user’s hand comes in contact with a virtual object: should it move through the object, or should the object block their movement much like it would in reality? While the latter option might make sense on paper, the fact that users could still move their physical hand while the virtual one stays stationary could result in confusion. The team moved beyond the challenge by letting users push virtual objects freely—for example, the monk model that they shave—which snap back into place once released (which sounds like a fun interaction in its own right).

The way that Hand Tracking SDK detects hands also presented a challenge: it seeks out the shape of a hand against a background, so it loses tracking once two of them overlap. “You can’t place a menu on the palm of your hand and tap an option on it, or interact with a virtual object on your wrist, for example,” says van het Kaar. To work around this challenge, a menu floats beside the user’s hand. While this doesn’t allow for haptic feedback by selecting options against one’s own body, this setup mitigates the risk of losing the tracking by having hands overlap.

Taking the time to experiment and apply these learnings allow us to develop increasingly realistic experiences in extended reality. From playing with hand tracking in VR to demonstrating how occlusion transforms experiences in AR, our team of makers are devoted to continually experimenting with new technologies, finding their most relevant use cases and establishing best practices for brands and our partners. As barriers continue to break down between the physical and virtual, it will be exciting to see what kinds of wholly new digital experiences emerge.

Get ready for an upgrade: in early December, Google revealed its Depth API, a new functionality coming to ARCore that allows virtual objects and real-world environments to play nicer together, allowing for more convincing and immersive mixed reality experiences. A demonstrable way that Depth API achieves this is by enabling occlusion, or the illusion of virtual objects’ becoming obstructed behind real-world ones.

Convincing occlusion has historically been difficult to achieve, though Google has put together a video portraying demos of the new API that show off its features. One of those demos, which challenges the user to a virtual food fight against a levitating robot chef, was developed in collaboration with MediaMonks.

What’s exciting about Depth API is its ability to understand the user’s surroundings at an unprecedented speed and ease. “The API’s depth map is updated in real time, allowing AR apps to be aware of surfaces without complex scanning steps,” says Samuel Snider-Held, Creative Technologist at MediaMonks. This enables not only occlusion as mentioned above, but also the mimicry of real-time physics. For our virtual food fight against the AR-rendered robot, missing is part of the fun; users can take delight in the digital splatters of food on the objects around them without worrying about cleanup.

The Building Blocks to More Immersive AR

How does Depth API work, and what sets it apart from other methods of occlusion? “The Depth API uses an approach called ‘depth from motion,’ in which ARCore determines distances to objects by detecting variances between image frames while the camera is moving,” says Snider-Held. “The result is a high-resolution depth map that is updated in real time, allowing the device to better understand where objects are in relation to one another and how far away they are from the user.”

Depth API is software-based, requiring no new hardware for users with ARCore-enabled devices once it releases publicly. While sufficient occlusion significantly increases the verisimilitude of virtual objects, it follows a series of incremental updates that build on one another to allow for more realistic immersive experiences. Just last year—the same year ARCore debuted—Google released its Lighting Estimation API, which lights virtual objects to match the existing lighting conditions in the real-world setting, including light reflections, shadows, shading and more.

Since then, a feature called Cloud Anchors allows multiple users to view the same virtual objects anchored in a specific environment. It’s the key feature powering the multiplayer mode of Pharos AR, an augmented reality experience we made in collaboration with Childish Gambino, Wolf + Rothstein, Google and Unity—which itself served as a de facto demo of what Cloud Anchors are capable of in activating entirely new mixed reality experiences.

“We have the creative and technical know-how to use these new technologies, understand why they’re important and why they’re awesome,” says Snider-Held. “We’re not scared to take on tech that’s still in its infancy, and we can do it with a quick turnaround with the backing of our creative team.”

A Streamlined Way to Map Depth

Depth API wasn’t the first time that MediaMonks got to experiment with occlusion or spatial awareness with augmented reality. Previously, we got to experiment with other contemporary solutions for occlusion, like 6D.ai, which creates an invisible 3D mesh of an environment. The result of this method is similar to what’s achieved with Depth API, but the execution is different; translating an environment into a 3D mesh with 6D.ai is fastest with multiple cameras, whereas Depth API simply measures depth in real time without the need of scanning and reconstructing an entire environment.

Similarly, Tango—Google’s skunkworks project which was a sort of precursor to ARCore—enabled special awareness through point clouds “When we had Tango from before, it used something similar to a Kinect depth sensor,” says Snider-Held. “You’d take the point clouds you’d get from that and reconstruct the depth, but the new Depth API uses just a single camera.”

In essence, achieving occlusion with a single camera scanning the environment in real time offers a leap in user-friendliness, and makes it widely available to users on their current mobile device. “If we can occlude correctly, it makes it feel more cemented to the real world. The way that they’re doing it is interesting, with a single camera,” says Snider-Held.

Adding Depth to Creative Experiences

Depth API is currently opening invitations to collaborators and isn’t yet ready for a public release, but it serves as a great step in rendering more believable scenes in real time. “It’s another stepping stone to reach the types of AR experiences that we’re imagining,” says Snider-Held. “We can make these projects without caveats.”

For example, a consistent challenge in rendering scenes in AR is that many users simply don’t have large enough living spaces to render large objects or expansive virtual spaces. Creative teams would get around this by rendering objects in miniature—perhaps just contained to a tabletop. “With Depth API, we can choose to only render objects within the available space,” says Snider-Held. “It lets us and our clients feel more comfortable in making these more immersive experiences.”

As brands anticipate how they might use some of the newest features of fast-evolving mixed reality technology, they stand to benefit from creative and production partner that can bring ideas to the table, quickly implementing them with awareness of the current opportunities and challenges. “We bring creative thinking to the technology, with what we can do given our technical expertise but also with things like concept art, animation and more,” says Snider-Held. “We don’t shy away from new tech, and not only do we understand it, but we can truly make something fun and inventive to demonstrate why people would want it.”

The decade is drawing quickly to a close, and it’s been a wild ride. From new technologies to new members of our family (we welcomed BizTech, IMA, Firewood Marketing and WhiteBalance this year), 2019 presented us with a lot of thrilling changes—and some exciting opportunities as we enter a new era. Looking back, we polled managing directors from our offices around the world for their favorite trends and technologies that have emerged in the past year—and what they’re looking forward to next.

Extended Reality Gets Real

Interest in mixed and extended reality (the combination of real and virtual objects or environments, like augmented or virtual reality, enabled by mobile or wearable devices) has been growing. At the same time, mixed reality has made strides in maturity over the past year, like Google’s efforts in making virtual objects feel truly anchored to the environment with occlusion, in which virtual objects are responsive to their surrounding environment—for example, disappearing behind real-world objects.

For Martin Verdult, Managing Director at MediaMonks London, extended reality is among the innovations he’s become most excited about going into 2020, and not just for the entertainment potential: “Virtual and augmented reality will become increasingly prevalent for training and simulation, as well as offering new ways to interact with customers.” For example, our Spacebuzz virtual reality experience gives children a unique look at the earth and environment they may typically take for granted, using the power of immersive tech to leave an indelible mark.

As the technology that powers extended reality matures, so will its potential use cases. But when a technology is still evolving significantly in short time, it can be difficult for brands to translate their ideas or goals into clear, value-added extended reality experiences. “We have introduced creative sprints for our core clients to get these ideas in a free flow,” says Verdult.

Among Verdult’s favorite examples of augmented reality projects MediaMonks has worked on this year is Unilever’s Little Brush Big Brush, which uses whimsical, virtual animal masks to teach children proper brushing habits and turn a chore into playtime. Similarly, extended reality can bring products to life in an engaging way—or if used in a customer’s research phase, it can help customers interact with a product with minimal (or no) dedicated retail shelf space.

Part of the Little Brush Big Brush’s charm is that it extends beyond simply AR, connecting to a web cartoon series and a Facebook Messenger chatbot to reward kids with stickers at key milestones. “Value comes from connecting an IP to a brand through a deeply engaging hyper reality experience,” says Olivier Koelemij, Managing Director at MediaMonks LA. “One that only a well-executed integrated production can offer, combining digital and physical in new and extraordinary ways.”

AI/Machine Learning Grows Up

One can’t reflect on past innovations and look to the future without mentioning artificial intelligence and machine learning. From programmatic delivery to enabling entirely new creative experiences—like matured extended reality powered by computer vision—to connecting cohesive experiences across the Internet of Things, artificial intelligence “will change our interaction with technology in ways we can’t imagine yet,” says Sander van der Vegte, Head of MediaMonks Labs, our research and development team that continually experiments with innovation.

The most creatively inspiring uses of AI are the ones that will help us understand the world and our fellow humans. In collaboration with Charité, for example, we programmed a 3D printer to exhibit common symptoms of Parkinson’s disease and its effect on motor skills. The result is a series of surreal art-objects that make real patients’ experiences tangible for the general population.

Social Content and Activations Build Impact

Ask Sicco Wegerif (Managing Director at MediaMonks Amsterdam) what struck him this year, and he’ll tell you it’s the elevation of social content in purchasing—for example, how Instagram made influencer posts shoppable early this year. Wegerif notes that about a quarter of consumers have made a purchase on social media, signaling new opportunities for brands to build connections with consumers.

“Looking at this from an integrated and smart production perspective, we can help brands create so many assets and storylines that tap into this trend, especially when combining this with data so we can be super personal and relevant.” When social media is prioritized early in the creative and planning process, it can enable more meaningful experiences.

For example, our “People are the Places” activation for Aeromexico used Facebook content to transform the way users discover destinations around the world. Instead of researching and booking a city, users get to learn about people around the world—then purchase a ticket to where they call home. The social content enriches the experience and builds emotion into the experience. “It’s in essence a very simple thought that can change the whole CX,” says Wegerif.

Social Activations and Digital Experiences Weave Together

Speaking of social media, it can become a powerful tool to build relevance and connection with experiential. Jason Prohaska, Managing Director at MediaMonks NY, says: “Experience and social work hand-in-hand as part of the digital plan for many brands, and are no longer below the priority line.” With live experiential—which elevates the role of the online audience to interact, take part in and build buzz around experiences—brands can achieve greater strategic impact in how they build connection with their consumers.

But doing so successfully requires a confluence of data, influencers, experiential storytelling and production. The future of this looks good to Prohaska. “We expect 2020 to deliver several use case scenarios at scale for brand identity that may set benchmarks for personalization, automation, customer journey optimization, efficacy, performance and engagement.”

Koelemij looks forward to stronger investment in digital and consumer understanding as brands begin to integrate experiences even further going into 2020. “With most good work, success and performance can now be better attributed to digital as we get more advanced in understanding what success looks like,” he says, “especially in how we can measure it across blended activations.”

And that’s exactly how we’d like to spend 2020: helping brands achieve their goals with data-backed, insights-driven creative across the customer decision journey. Through added capabilities thanks to companies like WhiteBalance, Firewood, BizTech and IMA joining the S4Capital family in 2019, we achieve this by greatly prioritizing and enhancing key elements of the marketing mix for daring brands—and as we reflect on the past year, we can’t wait to see what’s next.

This week, Fast Company launched its annual Innovation Festival, featuring Fast Track sessions that take attendees behind the scenes and into the homes of some of the most innovative companies in New York City. Billed as “Fast Company’s unique take on the field trip,” Fast Track sessions engage brands and creatives through hands-on talks and experiences hosted by participating companies, including MediaMonks.

Our New York office opened the doors to the dojo, inviting brands into our home to discuss all things extended reality. In a panel session devoted to augmented reality and its application to music and entertainment, our Monks dove deep into the design and development process of Pharos AR—a mobile AR experience made in collaboration with Childish Gambino, Wieden+Kennedy, Unity and Google. Taking users on a cosmic journey set against an exclusive track from Childish Gambino, the app is notable for being the first multiplayer music video.

With a panel including Snider-Held (Creative Technologist, MediaMonks), Thomas Prevot (Senior Producer, MediaMonks) and Tony Parisi (Head of VR/AR Brand Solutions, Unity), the session served as a casual fireside chat. The conversation kicked off by establishing the state of VR and AR, often characterized by the conflicting feelings that VR is dead and that the clear use case for AR hasn’t yet been found. But both technologies are well established, each excelling in achieving different goals within different environments.

Showcasing our Batman experience as a strong example of the immersive powers of VR, Snider-Held noted that “These experiences are still very installation-based,” and that AR’s distribution through mobile offers the potential for greater reach with a simpler experience. In explaining the process of developing Pharos AR in particular, the group explored key considerations for challenges to consider when developing an extended reality experience.

AR Can Feel Real Without Being Photoreal

Constraint prompts creativity—an adage that applies just as well to AR as any other medium for art making. Because mobile AR experiences are designed for use across a variety of devices, they must be relatively lightweight to provide a smooth experience to the widest share of users. Failure to keep technical constraints at top of mind can instead result in a lagging, stuttering experience that breaks immersion.

While this is true for any digital experience, it’s especially true for AR, a medium which Parisi says aims to “intelligently interact with the real world.” This expectation to seamlessly integrate with the surrounding environment can make stuttering graphics stick out like a sore thumb. “You want to keep the frame rate above 30 frames per second,” says Snider-Held, “because the user will compare the motion on the screen with the action happening around them in reality.”

The trio took this challenge as an opportunity to discuss the highly stylized look achieved with Pharos AR. While a photorealistic graphics might be impractical for a mobile device to realistically render in real time, a stylized look presents the opportunity to differentiate your experience through a strategic choice in aesthetic; for Pharos AR, the team took visual inspiration from Childish Gambino’s laser-punctuated stage shows, ensuring the app’s look and feel naturally integrated with the rest of the artist’s visual brand.

“Stylistically, we were trying to remain within the constraints of mobile processing in a visually appealing way,” said Snider-Held. An example of this is the use of particle effects, in which sparkles of light coalesce into a ghostly image of Childish Gambino as he dances to the music, animated via motion capture. “This is the best example on why you don’t need to do photorealism,” Parisi said. “We were able to capture the essence of Donald, because it’s his dance.”

Carefully Plan the Narrative Environment

Extended reality experiences are interactive by nature, meaning they rely on a different approach than how you would plot out and plan more linear experiences. There’s a careful balancing act between giving users the reigns to explore on their own versus stringing them along a narrative thread. MediaMonks Founder and COO Wesley ter Haar notes that “Narrative UI is key for onboarding and guiding the user in an AR experience,” making it incredibly important that you plan out users’ interactions and use environmental cues to shape a narrative.

While Pharos AR begins and ends through open-ended user interaction, it still follows a clear narrative through the virtual performance of Childish Gambino’s single, “Alogrhythm.” In exploring the primary path through the experience, the team began planning it in storyboard form, much like you would make for a film. “This process not only serves as visual research, but also in briefing the animation team and envisioning how actions will play out in the user’s environment,” said Snider-Held.

That makes sense—but what sets storyboarding for AR or VR different than other forms of digital storytelling? Because users could play with the app in their homes, the team had to plan for other variables how large the virtual scene should be, and what actions would be possible for multiple users to make within a smaller environment—like the cramped living room of a New York apartment, to offer an example that the Fast Track attendees could relate to. The challenge demonstrates how important it is to map out the virtual scene for different scenarios and users.

An interesting insight uncovered in the panel was that the team didn’t just rely on visual methods of planning like maps and storyboards. Due to the nature of the background music building up as users explore the space around them, the team also developed a musical timeline that maps up how different interactions trigger the layering of the music. The step showcases how sensorial, environmental cues can shape the action within an immersive, extended reality experience.

Whether developing for VR or AR, extended reality experiences require developers to rethink the creative approach beyond the standard linear story. From considerations in setting, technical constraints and variations in number of users across platforms, extended reality development relies on a comprehensive understanding of the building blocks that make up a total user experience. Snider-Held capped off the session with an ambition on what MediaMonks aims to achieve with brands through such experiences: “We strive to further the use of the technology from impossible to probable, and experiment in how to further that, too.”

Earlier this summer, we released  Pharos AR, a mobile app that takes Childish Gambino fans on a virtual, hallucinogenic journey through the cosmos starting from wherever they stand in the real world. Fans and press alike were impressed by the artist’s bold foray into virtual space.

VentureBeat was excited by the long-term value of the app, for example: “Between the cool visual effects and Glover’s music—which the app will apparently update with new songs over time—there’s certainly enough here to merit a download for fans.” Variety, meanwhile, applauded how well it integrated within the larger Childish Gambino universe: “The whole thing is very spacey, and stays true to Childish Gamibo’s other ‘Pharos’ projects” that have expanded across performances.”

As we see interest build in augmented and mixed reality over the interim, we’re looking back on the project’s development and how multiple parties and team members came together to employ best practices in UI and design, helping everyday users ease into a mysterious–and perhaps technologically overwhelming–new interface without limiting space for play and exploration.

Made in collaboration with Google Zoo, Unity and MediaMonks, it’s the world’s first shared augmented musical experience, allowing multiple users to enjoy a unique, artistic experience together. “This app is a breakthrough for AR,” says Thomas Prevot, Sr. Producer at MediaMonks. “It serves as another outlet for Childish Gambino’s creativity, letting him update his fans with future song releases over the cloud,” says Prevot, explaining how the app fits within the larger Childish Gambino brand.

Pharos AR also showcases the power of the ARCore platform and how it can enable immersive, social storytelling experiences. In particular, it shows off the capabilities of Cloud Anchors, which lets multiple users interact in a shared virtual space–which also makes it fairly unique among AR experiences. The tech’s newness can be intimidating, though: how can apps cultivate an interactive, exploratory experience for those new to AR?

Integrate Brand Familiarity & Digital Ecosystems

Childish Gambino fans will recognize Pharos AR’s light-particle silhouette from a Pixel 3 TV ad, and scenes projected within an enigmatic pair of monoliths at this year’s Coachella festival. These elements are purposefully and artfully executed across touchpoints, and this integration with a pre-existing digital ecosystem helps to make the process of designing for emerging tech more intuitive.

From the Pharos Festival to a TV spot to Coachella and beyond, you’ll find the same trademark, psychedelic motifs true to the nature of the artist. This goes to show the potential of AR as a powerful channel to not only engage audiences in an immersive way, but to bridge together the Childish Gambino gospel as well.

Get Intuitive

With AR still being fairly new to some, developers must think carefully about onboarding new users. “Experiences like Pharos AR provide an exciting opportunity for us to help make AR and VR more accessible to wider audiences,” explains Justina Sung, UX Designer at MediaMonks. “If people habitually use their phones a certain way, how do we break out of that to teach them new behaviors for emerging tech like AR?”

Inspired by Childish Gambino’s spirit of ambiguity and minimalism, the app integrates a sensory communication method, instead of verbal directions, to onboard. “Most app tutorials task the user with swiping through instructional cards, but we made it feel like the title sequence of a movie,” says Alex Otto, Associate Creative Director at MediaMonks. When users first open the app, for example, a pink laser beam dynamically snakes through space and into icons that illustrate next steps. “It’s designed to give off a mysterious feeling that builds up layers of suspense,” says Otto.

Direct Users Through Environmental Storytelling

Because AR rewards users through exploration, designing for it requires a careful balance between providing freedom versus direction. “It’s important that the design gives fans room to explore on their own to find meaning, using subtle environmental cues that nudge users in the right direction,” says Sung.

In Pharos AR, the scene gradually dims as users discover hidden glyphs on the cave walls. This nudges them to shift their focus to the altar, which lights up as users discover more paintings—a bit like a progress bar. Other cues include haptic feedback and the gradual, sonic buildup to Childish Gambino’s song “Algorythm,” which released through the app.

AR is a powerful platform for brands to tell their stories and engage directly with key audiences. In the case of Pharos AR, the technology provides users a chance to embark on a wild, immersive journey that encapsulates Childish Gambino’s message of enlightenment. As the first interactive, shared musical experience in AR, the experience truly pushes the limits on what can be achieved—inspiring brands and artists alike to consider what’s possible.

Eager to experiment with easily approachable AR?

Here, our friends at SoDA (or The Digital Society) sit down with MediaMonks Founder and COO Wesley ter Haar, who is also on the SoDA Board of Directors.

In this year’s Global Digital Outlook Study with Forrester, SoDA expanded their inquiry in the area of innovation labs to uncover some interesting findings.

SoDA: Not only are agencies continuing to launch internal labs and incubators but, more importantly, they are making direct investments into these initiatives. Do you think agencies are finally getting serious about innovation and realizing they need to invest in R&D rather than just hope to do cool work as part of regular client initiatives? How does MediaMonks approach innovation and do you invest in it outside of directly funded client projects?

Wesley ter Haar: I’ve never been a great fan of the Lab moniker. It’s a weird way to silo innovation as a play-thing instead of making it a core part of the day-to-digital-day we all live in. 

…The US is our largest market and the most ambitious, APAC leads the way in user behavior, LATAM has some of the most creative visual talent I’ve seen and I’m always amazed by the creative ideas that bubble up from smaller European markets. It goes to show that constraints are never a reason to deliver mediocre work.

This article was originally published in The SoDA Report – with key findings from Forrester.

Wesley ter Haar: That X used to be 24 to 36 months, and has probably been whittled down to 9 months with the constant change that besets consumer behavior and client adoption. At MediaMonks we hire or acquire against an internal innovation roadmap based on where we see the confluence of people, products and platforms are headed. For us, that has meant the acquisitions of a VR-first production company and a connected commerce company, the launch of a digital first content company and a hiring spree to bolster our AR capabilities. So, yes, innovation is critical to the health of our business, but I don’t believe a ‘Lab’ is the way to make it central to who we are and what we do.

SoDA: MediaMonks is hired by client-side marketers (and agencies) to deliver cutting edge work. This year we found that Chatbots/Conversational Interfaces, AI/Machine Learning and Programmatic Advertising topped the rankings for anticipated impact and planned investment in emerging technology. Agency leaders and marketers were generally aligned on this front with one major exception… Virtual and Augmented Reality. Marketers are planning to make significant investments in VR/AR while agency leaders are lukewarm on the short-term marketing impact. Why do you think there’s such a big gap between marketers and agencies on this front?

Wesley ter Haar: I think this gap mostly represents the excitement for the future state of “The R’s” (augmented reality, virtual reality and mixed reality) relative to the technical maturity of current platforms and production processes.

From the SoDA perspective, this shows a mature agency landscape with many agency leaders trying to think from a client perspective, focus on value/budgets and look at reasonable metrics. As agency leaders, we have to make sure we educate clients on the now while planning for the future so we don’t miss the boat when the scale and spread of these technologies starts impacting brands, business and bottom lines.

SoDA: For many years, SoDA has tracked what marketers value most in their agency relationships and, on the flip side, what agencies think their clients value most in their partnership with them. “Expertise in Emerging Tech/Trends” and “Process/Project Management” are consistently rated in the Top 5 by both agency leaders and client-side marketers. How does MediaMonks balance the importance of project management rigor with the desire for clients to explore (and quickly deliver on) the latest technologies? Is there a healthy tension between these two factors?

Wesley ter Haar: There will always be tension between doing difficult things for the first time and delivering difficult things for the first time, on time. Our role is to explain risk, mitigate against it as best we can, and make the “fall down and get back up” process of research, innovation and iteration one that is transparent to clients.

…But, the level of comfort on the client side is going to rely on the quality of the process and the project management rigor around it. In the same way that ad agencies are not artists, digital agencies shouldn’t hide behind labs and a “Crazy Scientist” vibe when it comes to new technology and trends. It’s all about the practical application for clients and their (potential) customers.

SoDA: This year we asked agency leaders to identify strategic factors they saw as most critical to their ongoing growth and evolution. Not surprisingly, “Attracting and retaining top talent” and “Developing new services / capabilities” topped the list. Interestingly, very few looked at “Expanding to new markets/geographies” as an important part of their strategic plans. MediaMonks appears to be quite the opposite with offices now in Amsterdam, London, Stockholm, New York, LA, Dubai, Sao Paulo, Buenos Aires and Singapore. How do you approach geographic expansion and why has it been so central to your growth strategy? What challenges have you wrestled with in managing the business across such a broad geographic footprint? What do you see as the most exciting new markets?

Wesley ter Haar: To start with the reasons, it gives us the opportunity to recruit and retain talent at a much larger scale, and in turn helps us cater to the ambition many of our Monks harbor when it comes to working in other countries and cultures. 

We run all offices as a single P&L which sounds like an admin choice, but is a critical cultural component. We are one company operating across 9 countries and 10 offices, with teams and talent working across time zones. Budgeting, resourcing and planning needs to be seamless to make that work, and that’s been the operational focus from Day 1.

VR and AR are the future, or so they say. With headlines like “2016 Will Be the Year That Sets the Stage for Virtual Reality” and “How VR Is Starting To Become Our Reality In 2017” taking over the hyper-saturated blogosphere, it might seem like VR and AR are the only technologies worth investing in.

But, as a virtual and augmented reality creative technologist, I’m constantly telling clients and colleagues to question this sentiment.

I work on some of the world’s most forward-thinking VR and AR projects every day and wholeheartedly believe in the power of these technologies to alter, integrate, or create new experiences and memories. At the same time, it’s also my job to think critically about technology and what new tools are best suited to meet client objectives.

VR struggles with two things: sharing and distribution. Since digital advertising lives or dies by social, the question many brands face is, how do we share cutting-edge VR and AR experiences? The struggle is that these experiences are not inherently shareable. VR hijacks your perception of the world by creating an illusion for you and your eyes only. Unless the VR experience is broadened through another channel, such as a teaser video on Youtube for example, if you want to share it with your friends, they have to be as big as a VR geek as you.

So if you’re a digital strategist or brand manager stretching yourself to explore how you can engage with these tools, I encourage you to stop, take a breather, and first read this post. It might just be that the best tool for the job is another technology entirely.

The Next Big Thing — Social VR Integration

Currently brands are only interested in creating their own VR experience, instead of exploring opportunities surrounding the VR hype. Brands creating their own one-off VR or AR experiences in hopes of creating a new channel for brand awareness is like creating an entire branded social network or community. This was very popular in the early days of social media advertising, but now you advertise within these social networks instead of trying to replace them. The same will happen with AR and VR.

For example, while Facebook is working on their experimental VR social platform Spaces, there already exists social platforms like the Rec Room and Altspace where users can virtually join others and talk, play games, and create things. If you’re hell-bent on creating VR and AR content for your brand, this is the trend to watch. The social iterations of VR will have an infrastructure designed for you to tell brand stories. This will be much cheaper and easier than creating your own application, and you can see the beginnings of this in Facebook’s AR studio. And similar to the way much of our current work is focussed on creating content for existing platforms, we’ll be delivering VR and AR in this way.

Experiences Unique to VR & AR

It’s undeniable that VR and AR can create unique experiences and express creative ideas that are not possible with any other technology. So if you’re dead set on creating a VR or AR experience, then make sure that you play to the medium’s strengths.

Take for instance this mixed reality case: ASM: Into the Wild, the world’s largest mixed reality experience to date. Using AR markers to place tiny virtual objects or characters on your table has been around for ages, but this is different. Using Google’s Tango technology, a museum was augmented into a living rainforest. Walls were transformed into trees and corridors into forest paths and guests were given tablets which they could walk around and interact with endangered animals, something they can’t do in real life. The magic of AR is not just bringing virtual animals to your dining room table, but to your entire environment.

Or perhaps my favorite example, Google Earth VR. Imagine having the whole planet at your fingertips, one moment deftly flying through the skyscrapers of midtown Manhattan, and the next sitting peacefully at the top of Mount Everest. Taking data from satellite imagery and 3D photogrammetry, the environments in Google Earth VR are majestic, and are some of the most presence inducing in all of VR. There’s nothing more mind blowing than virtually standing outside of your apartment looking up at your window, knowing that you’re actually inside, decked out in VR gear. Furthermore, the experience really gets you excited about the VR’s future. If this is what it looks and feels like now, what will it look like 10 years from now?

The Future

But what about other VR experiences that could well and truly be useful for real life? In the automotive industry you could allow potential customers to drive through impossible test drives, while changing features and testing during the same experience.

Or, think about how VR can provide an amazing tool for training new professionals in technical fields.

What if you could train to be a wind turbine technician by running through a variety of possible scenarios before you ever step foot in one? Or, imagine learning a language. How useful would it be to simulate the feeling of language immersion by placing someone learning french in a Parisian cafe, and they can only navigate the experience by correctly pronouncing various phrases?

Or museums! Imagine going to the MOMA and seeing Jackson Pollock via an AR tablet ferverishly throwing paint at one of his canvases!

The possibilities are endless, but that doesn’t mean that every possibility is right for your brand. So before you spend a whole lot of your (or your client’s) money, ask yourself this. Why do you want to create a VR or AR experience? Do these technologies really provide your brand something better than other technologies? Or is your idea just a gimmick? If you’re looking for reach or engagement, then maybe wait a while before reaching for a VR headset. As the VR and AR markets mature, the channels for telling your brand stories will mature with them.

This article originally appeared on Shots on July 5, 2017.