Earlier this summer, we released  Pharos AR, a mobile app that takes Childish Gambino fans on a virtual, hallucinogenic journey through the cosmos starting from wherever they stand in the real world. Fans and press alike were impressed by the artist’s bold foray into virtual space.

VentureBeat was excited by the long-term value of the app, for example: “Between the cool visual effects and Glover’s music—which the app will apparently update with new songs over time—there’s certainly enough here to merit a download for fans.” Variety, meanwhile, applauded how well it integrated within the larger Childish Gambino universe: “The whole thing is very spacey, and stays true to Childish Gamibo’s other ‘Pharos’ projects” that have expanded across performances.”

As we see interest build in augmented and mixed reality over the interim, we’re looking back on the project’s development and how multiple parties and team members came together to employ best practices in UI and design, helping everyday users ease into a mysterious–and perhaps technologically overwhelming–new interface without limiting space for play and exploration.

Made in collaboration with Google Zoo, Unity and MediaMonks, it’s the world’s first shared augmented musical experience, allowing multiple users to enjoy a unique, artistic experience together. “This app is a breakthrough for AR,” says Thomas Prevot, Sr. Producer at MediaMonks. “It serves as another outlet for Childish Gambino’s creativity, letting him update his fans with future song releases over the cloud,” says Prevot, explaining how the app fits within the larger Childish Gambino brand.

Pharos AR also showcases the power of the ARCore platform and how it can enable immersive, social storytelling experiences. In particular, it shows off the capabilities of Cloud Anchors, which lets multiple users interact in a shared virtual space–which also makes it fairly unique among AR experiences. The tech’s newness can be intimidating, though: how can apps cultivate an interactive, exploratory experience for those new to AR?

Integrate Brand Familiarity & Digital Ecosystems

Childish Gambino fans will recognize Pharos AR’s light-particle silhouette from a Pixel 3 TV ad, and scenes projected within an enigmatic pair of monoliths at this year’s Coachella festival. These elements are purposefully and artfully executed across touchpoints, and this integration with a pre-existing digital ecosystem helps to make the process of designing for emerging tech more intuitive.

From the Pharos Festival to a TV spot to Coachella and beyond, you’ll find the same trademark, psychedelic motifs true to the nature of the artist. This goes to show the potential of AR as a powerful channel to not only engage audiences in an immersive way, but to bridge together the Childish Gambino gospel as well.

Get Intuitive

With AR still being fairly new to some, developers must think carefully about onboarding new users. “Experiences like Pharos AR provide an exciting opportunity for us to help make AR and VR more accessible to wider audiences,” explains Justina Sung, UX Designer at MediaMonks. “If people habitually use their phones a certain way, how do we break out of that to teach them new behaviors for emerging tech like AR?”

Inspired by Childish Gambino’s spirit of ambiguity and minimalism, the app integrates a sensory communication method, instead of verbal directions, to onboard. “Most app tutorials task the user with swiping through instructional cards, but we made it feel like the title sequence of a movie,” says Alex Otto, Associate Creative Director at MediaMonks. When users first open the app, for example, a pink laser beam dynamically snakes through space and into icons that illustrate next steps. “It’s designed to give off a mysterious feeling that builds up layers of suspense,” says Otto.

Direct Users Through Environmental Storytelling

Because AR rewards users through exploration, designing for it requires a careful balance between providing freedom versus direction. “It’s important that the design gives fans room to explore on their own to find meaning, using subtle environmental cues that nudge users in the right direction,” says Sung.

In Pharos AR, the scene gradually dims as users discover hidden glyphs on the cave walls. This nudges them to shift their focus to the altar, which lights up as users discover more paintings—a bit like a progress bar. Other cues include haptic feedback and the gradual, sonic buildup to Childish Gambino’s song “Algorythm,” which released through the app.

AR is a powerful platform for brands to tell their stories and engage directly with key audiences. In the case of Pharos AR, the technology provides users a chance to embark on a wild, immersive journey that encapsulates Childish Gambino’s message of enlightenment. As the first interactive, shared musical experience in AR, the experience truly pushes the limits on what can be achieved—inspiring brands and artists alike to consider what’s possible.

Eager to experiment with easily approachable AR?

Raising awareness or building compelling content around destinations that are newly offered, remote or struggle with mixed perception can be a challenge for travel brands. But through emerging tech and improvements in mobile platforms, brands can provide experiences that let consumers explore far-flung locales without having to leave their homes, letting them intimately learn about a place through immersive storytelling. 

According to a report from Expedia Group, one third of Latin American Gen-Z and millennial travelers perform the entire travel shopping journey via mobile devices. The finding showcases the importance in offering mobile experiences that streamline the decision-making process through value adds, attention-seeking imagery and tie-in to social platforms.

Know Your Audience & Grab Their Attention

In essence, brands can better reach these young travelers through mobile moments: the points at which users turn to their devices to seek inspiration or answer a question. Brands can achieve this by adopting a mobile-first mindset when designing user experiences. Start by thinking about the context through which consumers engage with their devices.

Next Destination: Our 2019 Travel Trends Report

When Edelman sought to promote Hawaii as a destination, they targeted busy New Yorkers on their daily commutes—a key moment in which users are staring at their phones.  With an attractively redesigned MetroCard directing users to an elegant mobile site, the campaign intervenes with commuters’ daily grind to inspire a sense of wanderlust. Produced by MediaMonks, the mobile site lets users discover each of the archipelago’s islands in stunning visual detail before entering a drawing to win a free trip. In addition to its visual design, another strength of the campaign is how well it “gets” its local New York audience and situates itself within their daily routine.

Innovate through Scalable Experiences

With competition shifting its focus to mobile as well, it can be a challenge for travel brands to stand out. But investing in emerging technologies or supporting new social platform features provide opportunities for brands to reach consumers in fascinating and noteworthy ways. Take airline brand KLM for example, which is no stranger to using mobile AR to transport users to another place.

MediaMonks worked with the brand to innovate even further by offering AR advertising directly within Facebook’s newsfeed—a feature that isn’t yet available to the public. The ads invite users to open a virtual door as they scroll past, which activates a 360-degree photo allowing users to view landmarks from different angles by moving their device. The immersive content focuses on new or lesser-known destinations serviced by the airline brand, making use of the technology to drive awareness and allow users to almost literally stumble into a brand-new environment.

A similar example is STC’s “Unveil Saudi” series of VR experiences. STC isn’t a travel brand—the content serves to showcase the strength of its network—but it does an excellent job of letting users inhabit remote landmarks in stunning 360-degree video. Users can enjoy the content by strapping their device to a VR headset, or simply drag their cursor to control the view on desktop. This versatility showcases an important consideration for immersive storytelling: it’s easily scalable and accessible to audiences.

Reap the Full Potential of Mixed Reality

The strategies above are great for raising awareness about destinations or pique consumers’ curiosity about a place. But what better way is there to showcase a travel destination than by giving consumers the opportunity to really feel as if they were there?

A best-in-class example of bringing the destination directly to consumers is the 4D trishaw ride in VR that we developed with TBWA for the Singapore Tourism Board. Users strap into a real trishaw equipped with a VR headset, which transports them to Singapore’s hustle-and-bustle. The trishaw makes for an excellent vehicle—excuse the pun—for such an experience, as participants inhabit the role of a passenger.

By focusing their gaze in different directions, users can choose the path they want to take. Along the way, spatial sounds, vibrations and scents provide a multi-sensory, immersive experience that makes users feel like they’re really there. This type of experience won’t be practical for every brand; as an installation, it makes a better fit at trade shows, for example. But travel brands that want to make an event out of highlighting a destination (like a tourism board) can benefit greatly from the technology.

Improvements in emerging tech and consumers’ favorite social networks provide new opportunities for brands to connect with audiences and inform them about their services or destinations. By letting users inhabit or explore an immersive location—whether it be just a quick moment on their phone or a lengthy drive through a city in VR—brands can raise awareness in compelling ways and increase their digital maturity.

Facebook hosted its annual F8 conference this week, announcing several new features on the horizon both big and small. With a focus on more responsible uses of tech and a rejuvenated focus on fostering communication between friends and loved ones, this year’s event felt a bit like the start of a new era—helped by the surprise release of an entirely new mobile app design.

Facebook’s vision has always been to bring people together, and at this year’s F8 conference the company placed its focus away from the News Feed and toward more genuine forms of communication, like Groups, Messenger and Stories. These features not only help friends and loved ones connect in new and more engaging ways, but can help brands engage more directly as well.

Offering More Direct Forms of Communication

Let’s start with Groups: Facebook killed its standalone Groups app nearly two years ago, but this week unveiled a new core app design that places Groups at the forefront, promoting them in different areas of the platform. This means users might find buy and sell groups promoted when exploring the Marketplace section of the app, for example. In addition to providing better visibility, Facebook is also enabling features specific to group types, like a template for employers to easily list job openings in groups for job seekers.

Facebook has also showed off more one-to-one types of interaction designed for smoother socializing and forming new relationships. Its Meet New Friends feature introduces users within shared communities, while its Dating feature offers a Tinder-like system that brings together people who share romantic interest in one another. Over on Messenger, meanwhile, friends can hang out virtually by watching videos together or even work together with a desktop app that allows for multitasking.

What’s interesting with these announcements is that the role of the News Feed—or at least broadcasting updates to it—becomes downplayed as Facebook explores other ways to bring people together through more direct forms of communication.

Chatbots are Invading Instagram

Chatbots aren’t new—Facebook popularized the medium at its 2016 F8 conference—but they’re about to get a lot better on Messenger. The most notable new feature is integration into Instagram ad units. Brands can include CTA’s in their Instagram ads that encourage users to swipe up to chat with an associate. A bot can handle the earlier stages of the conversation to qualify leads, then pass those leads along to a live agent. “It’s the perfect balance of scaling conversations for more basic interactions, while ensuring that qualified leads and complex customer service interactions are handled by real people,” says Nick Fuller, SVP of Growth at MediaMonks.

Another interesting thing about this functionality is how it brings different platforms in the Facebook family together: users may initiate a chat on Instagram, but the conversation happens within Messenger. “What’s important here is that Facebook is discovering more and more integration points of customer data and experience across their platforms,” says Fuller. “This means brands have the opportunity to target on one platform but easily retarget on another.”

The integration with Instagram ads highlights the success that brands have been having with its Stories format. “Stories is a really high-performing ad space for Facebook,” says Fuller. “With this chatbot integration into Instagram ads, brands can take users through the funnel with targeting, lead qualifying and conversion happening in one seamless flow—which is incredible,” said Fuller.

Bring Offline Spaces to Life

Facebook Camera Effects are cool and all—we wrote the book on it—but Facebook is looking beyond photo sharing to explore other opportunities where AR provides value. For example, users could scan a poster that transforms it into a three-dimensional scene or model, whether it be a portal that looks inward or an object that spills out of the surface, beyond physical constraint.

Fuller sees endless creative possibilities for brands that want to digitally engage with their customers within a physical environment. “This will be a killer feature for brands to reach users in retail or event spaces, for example” he says. This can range from fun moments (digital scavenger hunts that encourage you to explore an amusement park) to practical (a tutorial, viewed from multiple angles, showing how to put furniture together).

“From a product education standpoint, having the ability to aim your camera at a sign to achieve this is going to be a really helpful next-level AR capability.” We’ve seen how harnessing emerging technology can have huge benefits to getting customers to explore brick and mortar retail, so we’re excited to see how new leaps in accessible, scalable AR will further transform the physical shopping experience.

In addition to the features mentioned above, Facebook reiterated a couple key themes of the F8 conference: responsible use of technology (by combating fake news, making AI more inclusive and more) and a shift away from being a “social network” to a “social platform.” The new era for Facebook looks perfectly structured for enabling direct communication between brands and their fans, and we can’t wait to see how the platform further shapes up in the next year.

Can’t wait to dazzle Facebook users with SparkAR?

The camera is the new browser. Augmented reality has steadily improved over the years, allowing for entirely new, immersive experiences that remove the friction between brand offerings and their audience. The tech’s maturity is of particular interest for brands to add to their marketing mix thanks to its reach: through AR, any part of the user’s direct environment becomes a surface to advertise and build relationships.

So when I say the camera is the new browser, what I actually mean is that this emerging technology requires organizations to rethink how they tailor their message to their audience, just like with social media before it. We’re very bullish on AR, believing it will change the way people interact with the world and with each other, quickly and at scale. Brands should be a part of this process, helping to create those experiences—and lucky for them, there are ample opportunities for AR to create real business value. If you’re just wrapping your head around the new possibilities of AR or have considered investing in the tech for some time, consider these three questions early in your design process.

Why Do Consumers Love AR, and Where Do We Fit?

Augmented and virtual reality hit mainstream consciousness around the same time, making it easy to confuse the two. Throw in mixed reality and the differences between the platforms becomes even fuzzier. Which reality fits your idea depends on whether you want to tell a story to the user, or prefer to give them the tools to tell their own. If the first option is your goal, then VR or mixed reality is best. But when it comes to the second, AR makes the perfect environment for expression and experimentation.

While web AR still hasn’t launched, we were able to create a bit of a stopgap solution with our Dragon Spotting experience to promote Disney’s film Pete’s Dragon. While the web-based experience doesn’t load a model directly in the space around users, it prompts them to seek out specific environments—the family TV, for example—and pulls from the Google Cloud Vision API to identify whether the user has found it. If there’s a match, an animation of the dragon triggers to show that the user was successful. When Web AR really kicks off, such AR experiences will become much more sophisticated and allow for mass adoption.

Augmented reality is an exciting platform that asks both users, brands and developers to really reconsider the way they interface with machines. The technology is ever evolving, and brands shouldn’t wait to invite the platform into their strategies. By understanding what users love about AR, what the current limitations are and where it’s going next, organizations remain ready for this next big revolution in tech.

AR isn’t the only big trend to look forward to this year.

The most effective uses we’ve seen recently are with social platforms like Facebook and Snapchat, which have integrated AR filters into their apps’ respective camera features. This provides fun, out-of-the-ordinary opportunities to photograph and share (think funny facial effects or whimsical monsters joining them on the sofa) that fit seamlessly within the platforms. We’ve seen time and time again in this environment that AR is an instinctual medium: people react quickly and are comfortable in exploring, trying out and testing its features without the feeling of frustration or confusion that can be typical of entirely new interfaces.

As far as user experimentation is concerned, AR is very effective in mapping out information and allowing the user to explore or dive deeper however they choose. Why look at a tough-to-read diagram labeling out info when you can view it directly hovering above the object right in front of you? We set out to achieve this with an AR app that helped customers get acquainted with the hottest features of Weber’s Pulse grill. The app provided a virtual layer of informational hotspots that explained the grill’s parts and showed how they worked with animations. For enterprise use, this speaks to the original promise of AR: that users can easily see contextually relevant information without having to use their hands (i.e. like a car’s parts labeled in AR as a mechanic works on it).

On the consumer side, this method makes for a memorable example of persuasive marketing. With the Weber Pulse app, users had the ability to grill a virtual meal via AR first, alleviating any concerns before trying the real thing. By mimicking the real-life act of grilling BBQ, their experience with the product “sticks” better in their mind than if they’d just watched an ad or seen the grill on a shelf. This improves the chance of conversion or whatever KPI you’re tracking.

How Do I Anticipate and Capitalize on AR’s Limitations?

AR may have settled into broad appeal over the years, but it’s still an emerging technology with room to grow. Early in the process of concepting your experience, take the time to understand and embrace the limitations of the platform you choose. This is when you need to assess the specific platform that best supports the AR experience you want to provide: should you develop for a Facebook or Snapchat filter? Is a custom, fully-featured mobile app a better choice?

There are pros and cons to everything. Photo filters on Facebook and Snapchat both enjoy robust engagement and wide reach thanks to their integration with social platforms. AR in this environment is usually only mildly branded, where the focus is less on the product or brand and more on fun, shareable content that somehow aligns with it. But these filters’ ability to instantly load comes at a cost: a restrictive file size that limits functionality. Initially, this seems like a challenge. If you recognize this limitation and design a very focused experience around it, you can end up with an enthralling, snackable experience that fits the platform so well.

A dedicated AR app, meanwhile, provides greater functionality and creative opportunities through its integration with ARCore and ARKit in a mobile OS. While this platform allows you to do much more (like the Weber grill app discussed above), it’s costlier in resources and has a higher barrier of entry for consumers who may be less willing to download an app. That said, dedicated AR apps make excellent tools for retail events, trade shows or public installations, like the narrative AR experience we made with Santander.

Where is the Technology Headed Next?

New platforms for hosting and developing AR experiences will aim to reduce friction between brands and users, which is really the big promise of AR: providing a direct and intuitive interface that blurs the line between physical and digital. Brands who wish to advertise through AR will need to evolve with new possibilities and formats that emerge.

To get an example of an AR platform emerging right now, check out Google Playground, which is directly integrated with Google’s Camera app—at least for some users. The Playground feature offers a new type of interaction: rather than simply pose with a filter, users can freely place virtual objects in their environment before snapping a photo to send. They’re a lot like the stickers that are common in several photo editing apps, except these objects (called “playmoji”) are three-dimensional and stay in place as users move the camera around.

AR is still maturing, a point I touched upon at CES panel on where the AR experience is going. The next big leap for augmented reality will be web-based AR, something that Google has shown but isn’t yet widely available. In the meantime, the startup 8th Wall has tried to carve out a unique space in being the only platform for developing true, web-based AR experiences. Loading up AR experiences directly in a mobile web browser will dramatically reduce friction for the user, doing away with the need of downloading apps or navigating through a library of effects. Instead, you need only point them to your website to activate. With 5G on the horizon set to unlock connection speeds, users will soon have unprecedented access to more immersive experiences.

With omnipresent technology on the horizon, are we headed toward a cyberpunk dystopia?

That was the big question MediaMonks posed to the audience at its Fast Co. Fast Track session, offering attendees food for thought to mull over with drinks. Rejecting a future in which pop-ups and banner ads vie for our attention literally everywhere we look, speakers Eric Wagliardo and Jouke Vuurmans offered a more optimistic view. By using emerging technology like VR, AR and artificial intelligence to delight audiences, Wagliardo told the audience, “we can use creativity unconstrained by the past and the laws of physics.”

The theme of the talk anticipates a near-future in which wearables or ubiquitous AR will revolutionize the way we interact with media and technology, just like how the iPhone ushered in the era of smartphones. As these technologies become more consumer-friendly, brands might be biting their nails wondering if they’re ready for the new digital landscape. Are you?

The focus of your VR or AR experience doesn’t have to be grand to be impressive.

One of the more compelling details from the talk was about a humble plank of wood used in the Jack Ryan Experience, a 60,000 square-foot activation that pulled together some of the most cutting-edge VR technology to provide a 4D, immersive experience. Through a VR headset, the plank transformed into a startling chasm for participants to cross in virtual reality. That such a basic material in reality could become an intimidating environment to traverse demonstrates the power to create compelling experiences from just a simple object. With so many elements and gorgeous tech to compete with, who would expect a plank of wood to steal the show?

VR can be tricky, though: without a proper “hero device” like the iPod or iPhone to put the tech in almost everyone’s pockets, VR experiences must often be tethered to a specific environment for participants to don a headset and enter your world. This can be a big investment in terms of reserving a space as well as providing a rig to host the experience, though it’s well worth it if you’re hosted within a venue that allows you to reach your audience in a meaningful way. For some brands, a simpler solution might be to create an AR experience that participants may enjoy via nearly any mobile device.

Another benefit to AR is that it can enable a charming connection with objects or pre-existing campaigns that wasn’t possible before. One of the demos featured at the Fast Track event invited attendees to scan an ad for Kiwi shoe polish featuring a Van Gogh self-portrait, which prompted the artist to leap from the poster, ready to try on several pairs of shoes with the user.

The AR app was a remix of an award-winning Ogilvy campaign that “completed” various famous portraits by adding views of the subjects’ shoes. Partnering with Ogilvy and Google, MediaMonks was able to bring the Van Gogh portrait to life by stepping into the real world and responding to different shoe styles with animations and props. The experience of helping the famous artist try on shoes is humorous and helps the user relate to an esoteric character they certainly recognize, but may not have previously understood.

AR and VR are more consumer-friendly each day, offering several ways for you to reach them with ease.

 A large-scale project with emerging tech can be intimidating. While it can be difficult to identify where to begin planning bigger projects, there are several levels of engagement to design for in VR and AR, allowing brands who want to dive into emerging tech to start small and provide increasingly immersive experiences from there. One way to provide an engaging experience is to focus on one simple, primary mode of interaction. For example, the Kiwi case demoed at the Fast Track event required only a poster and a mobile device to render the character on the screen, along with some options for interacting with that character. This simple template could be useful for brands new to the emerging tech space, who also want to provide engaging, emotional experiences.

Seeing is believing.

And just like how the app brings the print ad featuring Van Gogh to life, you can use your app to remix existing assets from current or past campaigns, thereby boosting their ROI—not bad for a budget-strapped brand who wants to dive in on AR.

If a business has a little bit of extra room in their budget, they can elevate the same experience detailed above by staging it in an appealing way. While the primary mode of interaction might be to bring a character, object or other experience into the real world via AR, offering a set for participants to play with the experience can further immerse them. In the case of the Kiwi app, MediaMonks constructed a tiny room at Cannes Lions International Festival of Creativity decorated to look like a city street you might find in a Van Gogh portrait, allowing participants to step into the artist’s world (without the need for a VR rig and other expensive tech).

If one theme is clear across the Innovation Festival, it’s that the quickening pace of new media and platforms are challenging brands to think outside the box in providing new, experiences to enchant and inform their audiences. We were happy to allow guests into the hallowed chambers of the MediaMonks monastery to impart a secret or two about our process. But can the students surpass the master? We look forward to seeing what you create!

If you have any interest in tech, chances are you’ve watched an episode or two of Black Mirror, Netflix’s massively popular series that dives into uneasy relationships between people and tech (and if you haven’t, we suggest you start). While the drama sets its sights in the near future, its power—and proficiency for inducing anxiety in viewers—stems from the fact that the questions it explores are just as applicable to the present day.

If you take solace in the fact that Black Mirror’s terrifying tech hasn’t yet (or never will) come to fruition, don’t get too comfortable: the future is now, they say, and the mad minds at MediaMonks Labs are releasing a web app that brings the show one step closer to reality.

In the season 2 episode “White Christmas,” a neural implant called Z-Eye provides an augmented reality interface within a user’s field of vision. Among its features is the ability to block other people in real life, which makes them appear as gray silhouettes with muffled speech. Inspired by the feature, MediaMonks Labs built an AR filter that likewise renders a user’s body as a pixelated, black-and-white shape (try it here–best enjoyed on a tablet or laptop).

While it’s not quite as advanced as what you see on Black Mirror—it doesn’t require an implant, thankfully, and works with only one person in the view—it demonstrates how technology featured in speculative fiction is closer to reality than audiences might anticipate. In this case, the technology is so accessible that it requires little setup for users to play with. “What makes this so cool is that it’s running in your browser and any out-of-the-box webcam,” says Joe Mango, Creative Technologist at MediaMonks Labs, who built the tool. “It’s clickable and usable at any time.”

Block Out Anyone—No Implant Required

The filter is powered by BodyPix, an open-source machine learning model that enables body-part segmentation. “Segmentation” is a process in which a machine takes an image or video and separates the pixels that belong to a person versus those that do not. It’s the same idea behind using a green screen, no screen necessary.

What makes BodyPix unique is that it not only separates a body from its surroundings, but can also segment that body into 24 specific parts; for example, the left side of the face versus the backside of a subject’s right arm. Such technology could have several applications, like creating precise, body-tracking augmented reality filters or image editing.

With BodyPix as an underlying machine learning model, the Labs-developed body-blocking tool isn’t TV magic. And even though the technology that makes it work is quite complex, the concept behind body blocking is relatedly simple.

Rendering formless, pixelated bodies in real time, the Labs-developed body-blocking tool isn’t movie magic. The tech powering it is quite complex, but the concept behind body blocking is rather simple and relies on editing two identical video feeds together. On one end, the tool takes the camera feed and applies a shader that distorts and pixelates it. On the other end, a separate version of the video is segmented to cut out the user’s body—as if you took the magic wand tool image-editing program to select an object and delete it. This version of the video is pressed on top of the distorted one, resulting in a pixelated, black-and-white body surrounded by an otherwise normal environment.

Finding the Science Behind Science-Fiction

While the tool isn’t totally accurate to the technology as seen in the show, its underlying mechanics can help viewers understand and envision a technology that previously may have seemed possible only in the realm of science fiction. It also demonstrates the value of trying to make the impossible possible through cutting-edge and emerging technology.

“It may seem unusual that a creative and production company like MediaMonks is doing such in-depth technological research,” says Mango, though that openness to experiment can help brands find new opportunities to take experiences available in their worlds and replicate them for fans in ours. When Mango stumbled upon BodyPix in GitHub, for example, “My mind immediately went to the Black Mirror episode.”

The pitch to build upon the software to reproduce the startling Z-Eyes feature was surprisingly easy: “When I approached [Head of Labs Sander van der Vegte] with the idea, he said, ‘That sounds cool, let’s do it!’” recounts Mango. “In a nutshell, it’s that simple to get sign-off on a project thanks to the open, creative environment.” But considering Black Mirror is often a case study in troubling uses of technology, this zeal to experiment shouldn’t be viewed as flippancy. “We’re able to peer into Pandora’s box to see the interesting possibilities that are available—and the questions they raise,” says Mango.

And that ties back to how such projects can help end-users better apply fictitious, futuristic tech they see in the media to their own lives—and to anticipate or wrangle with some of the ethical dilemmas that inspire speculative fiction. And of course, such projects are just plain cool, too. In essence, it all boils down to making a connection that breaks down barriers between the real and imagined.

There’s no denying that online retailers make shopping more convenient, but in some ways the digital shopping experience falls short. For one, you can’t try on clothing or fashion accessories before having them shipped to your door, which means you might discover after days of waiting that something doesn’t fit. To get around this, customers might buy several versions of an item in different sizes, which can be costly to them in the immediate term while contributing to scarcity of high-demand items—for example, sneaker releases, which have generated $20 billion in U.S. sales alone in 2017 and have a resale market valued at over $1 billion.

There has to be a way to instill purchasing confidence in consumers, which is why one of the latest projects to come out of MediaMonks Labs, our internal R&D team, is an AR tool that uses computer vision to accurately measure shoppers’ feet. While it may not sound like the most exciting use of AR (no, it won’t transport virtual monsters to your living room or render a shoe directly on your feet), “it speaks a lot to what else we do beyond the crazier stuff,” says Joe Mango, Creative Technologist at MediaMonks Labs. “We explore things where there’s need, and see how feasible—or infeasible—it is. There’s high interest in exploration and discovery.”

Sander van der Vegte, who leads the Labs team, discusses the importance for more practical uses of the tech in both its adoption and signaling value to consumers: “Everyone’s pushing AR, but why is it useful? This tool is designed purely to solve a specific problem, and is functional in doing so.”

Improving CX with a “Form Follows Function” Approach

While most people are aware of their shoe size, fits aren’t so cut and dry; shoppers often get some shoes in one size and some in another. “Shoe shopping becomes more complicated when different brands and shoes have different sizing charts, making it harder to gauge what fits,” says Mango, who brought the tool to life. It can get around this issue by automatically translating a unit measurement into a specific brand’s sizing chart, giving the perfect size recommendation as users shop online. Ideally, the tool could exist within a retailer’s app, seamlessly aligned with the browsing process.

The tool has practical value for both consumers and retailers alike: shoppers know their purchase will fit before they buy it, and retailers don’t have to foot the bill of processing returns. This also means customers don’t have to worry about being blacklisted from making frequent returns, either—a fail-safe measure some retailers have used that erodes consumer trust and confidence. But perhaps most importantly, the tool serves as example for how retailers can use emerging technology to give consumers an obvious reason to provide their data. In this case, the length of their foot improves and optimizes the customer’s journey by enabling them to make the right purchasing decision.

Improving Measurements with Computer Vision

It’s worth noting that measuring via AR isn’t an entirely new capability: Apple, for example, integrates an AR “tape measurer” into the current version of iOS. While the feature is great for measuring perfectly straight lines or edges, it’s not the most accurate tool of measuring the length of a foot: there’s a lot of user error in moving the phone and finding the right angle to measure the foot.

The tool built by MediaMonks Labs is powered by OpenCV (a programming library for computer vision) and takes a different approach. Starting with a top-down photo of the user’s foot, it uses edge detection to determine and outline its shape. It then compares the size of that shape to a reference object whose size is defined and absolute—for example, a ubiquitous coin—which results in a consistently accurate measurement. While the output is in units of inches or centimeters, the tool has the potential to translate these measurements to brands’ custom sizing charts, taking the guesswork out of determining what fits.

Taking the Next Steps in Consumer-Friendliness

While the value of the AR tool is clear, added steps can be taken to improve usability. “There is a caveat to this method of measurement,” says Mango. “You need a flat color-contrasted background for accuracy,” which helps the computer vision model detect the foot’s edges.

Mango also discovered that the tool had difficulty differentiating a foot from the rest of a leg. The solution was simple: by wearing ankle socks he could segment the two with consistent accuracy. These limitations, along with the requirement of a universal reference object, present opportunities to make the tool more user-friendly for commercial use.

Despite its current limitations, the tool shows how brands might identify common issues on both the business and consumer side of the equation, then implementing technology in new creative, new ways to solve them. In the immediate term, these unique applications can provide the retailer with brand equity through emerging technology’s novelty. More importantly, though, they can deliver a better online or offline shopping experience to customers.

While “Celebrated with Champagne” may be one-third of our moniker, and an important milestone for much of our work and team, you won’t find too much of it on the blog. However, this accolade meant too much to not mention, and we’re proud to be in good company.

Keep reading for an excerpt of the announcement from Working Not Working (the full article is linked to below) –

Since launching a new and improved Working Not Working at the start of 2017, we’ve seen our community’s curated talent pool more than double.

With an increasingly formidable crowd of tastemakers and influencers on hand across the globe, we’d be foolish not to listen to them and selfish not to share what they have to say with you. WNW is all about bringing the highest possible caliber of creative talent to studios, agencies, tech giants, startups, brands, and nonprofits. But we’re equally invested in introducing the most forward-thinking companies to our ever-hungry creative talent.

That’s why we surveyed our members for the fourth year in a row, asking them which companies they’d kill to work for full-time. So we know who wants to work where, and what matters most to them when considering a job. Also, many of our members are fervent in their desire to maintain their freedom, which provides added weight and a litmus test for identifying only the companies that earn the most enthusiasm in creative circles.

Below, we list the top 50 most popular companies out of the more than 600 receiving votes. But first, a bow to the royalty of this survey. Those whose names have graced the results all 4 years. You know who you are. And so should everyone else: 72andSunny, Airbnb, Apple, Barton F. Graf, BBDO, Buck, Disney, Droga5, Google, IDEO, Mother, NASA, Nike, Patagonia, Pentagram, Pixar, Sagmeister & Walsh, SpaceX, Tesla, and Wieden+Kennedy. Make sure to check out the 2014, 2015, and 2016 results. We’d also like to quickly note that what our members possess in creative capacity they lack equally in spelling skills. One of the most popular companies was also the most consistently misspelled. Congratulations to Weden+Kennedy, Weiden+Kennedy, Widen+Kennedy, Wiede + Kennedy, plus Wieden-Kennedy for all making the list.

Next, a warm welcome to the following companies making the list for the first time: Amazon, Chandelier Creative, Collins, Everlane, Instagram, Mailchimp, MediaMonks, and Oddfellows. You’re about to become a lot more popular on the internet. We predict that soon “Amazon” and “Instagram” will become household names.

This is just part of the announcement posted by Working Not Working.

We’ve been called on many times to bring truly crazy ideas to life. From creating a robotic squid to a sandbox that turns into a VR race course, to bringing a rainforest to life with the biggest AR experience ever created.

And those were only this year.

We take great pride in being hailed as the industry’s go-to player for ambitious digital ideas, but we never set out to be the most innovative. We don’t have a “Chief Innovation Officer” or an “Innovation Department.” It’s never been our goal and yet, after 16 years in operation, having made the first Cannes’ Lions Innovation List in 2013 and with over 100 Lions to date, something seems to be going right.

Many ask us how it’s done and the answer is pretty simple: it’s saying yes to crazy requests for almost two decades.

We’d never built a remote robotic squid before, but we had built over 100 VR installations and all sorts of connected game experiences. So, when our friends at Wieden+Kennedy called on us to build a giant squid for Old Spice that would invite users to remotely control a tentacle to complete random “quests,” we had the confidence we could do it and threw everything we could at it.

There wasn’t a single part of this project that didn’t push us to our limits. From creating squid ink and inflatable respiratory sacks to creating a platform that could allow remote players to control a single tentacle to perform tasks in real-time. A lot of this was new for us, but the teams from across multiple offices and disciplines collaborated to push themselves to the limit and make it happen.

We’d never turned a museum into a rainforest, but Google Zoo challenged us to create the largest augmented reality experience on the planet, so we said yes. Making sure the walls aligned with the trees and the corridors with the forest’s paths wasn’t easy, but through a process of experimentation we developed the second ever experience created using Tango, Google’s new technology platform for computer vision, and transformed 10,000 square feet into a endangered rainforest.

So I guess it was only a matter of time before we were asked to launch a sandwich into space. Right? At this point there’s not a lot of wild ideas that feel out of scope. The sandwich launched this week. On the cover of the New York Times, it was featured in an article called “A Chicken Sandwich Hitches a Balloon Ride to the Stratosphere.” To call this ambitious is an understatement.

Why do we keep getting these briefs? When it comes to some of the more bizarre ideas, it’s often the case that although we haven’t delivered the exact project before, our clients know we can draw from our wealth of experience to give it our best shot.

We work with some of the world’s best agencies, and even for them some of these borderline insane projects might be a one-off experience. For us, we’re learning, practicing and refining our craft on a daily basis. The more shots into the stratosphere we’re asked to take, the bigger the chance of getting us, and our clients to the moon.

This article originally appeared on The Little Black Book on July 3, 2017.

VR and AR are the future, or so they say. With headlines like “2016 Will Be the Year That Sets the Stage for Virtual Reality” and “How VR Is Starting To Become Our Reality In 2017” taking over the hyper-saturated blogosphere, it might seem like VR and AR are the only technologies worth investing in.

But, as a virtual and augmented reality creative technologist, I’m constantly telling clients and colleagues to question this sentiment.

I work on some of the world’s most forward-thinking VR and AR projects every day and wholeheartedly believe in the power of these technologies to alter, integrate, or create new experiences and memories. At the same time, it’s also my job to think critically about technology and what new tools are best suited to meet client objectives.

VR struggles with two things: sharing and distribution. Since digital advertising lives or dies by social, the question many brands face is, how do we share cutting-edge VR and AR experiences? The struggle is that these experiences are not inherently shareable. VR hijacks your perception of the world by creating an illusion for you and your eyes only. Unless the VR experience is broadened through another channel, such as a teaser video on Youtube for example, if you want to share it with your friends, they have to be as big as a VR geek as you.

So if you’re a digital strategist or brand manager stretching yourself to explore how you can engage with these tools, I encourage you to stop, take a breather, and first read this post. It might just be that the best tool for the job is another technology entirely.

The Next Big Thing — Social VR Integration

Currently brands are only interested in creating their own VR experience, instead of exploring opportunities surrounding the VR hype. Brands creating their own one-off VR or AR experiences in hopes of creating a new channel for brand awareness is like creating an entire branded social network or community. This was very popular in the early days of social media advertising, but now you advertise within these social networks instead of trying to replace them. The same will happen with AR and VR.

For example, while Facebook is working on their experimental VR social platform Spaces, there already exists social platforms like the Rec Room and Altspace where users can virtually join others and talk, play games, and create things. If you’re hell-bent on creating VR and AR content for your brand, this is the trend to watch. The social iterations of VR will have an infrastructure designed for you to tell brand stories. This will be much cheaper and easier than creating your own application, and you can see the beginnings of this in Facebook’s AR studio. And similar to the way much of our current work is focussed on creating content for existing platforms, we’ll be delivering VR and AR in this way.

Experiences Unique to VR & AR

It’s undeniable that VR and AR can create unique experiences and express creative ideas that are not possible with any other technology. So if you’re dead set on creating a VR or AR experience, then make sure that you play to the medium’s strengths.

Take for instance this mixed reality case: ASM: Into the Wild, the world’s largest mixed reality experience to date. Using AR markers to place tiny virtual objects or characters on your table has been around for ages, but this is different. Using Google’s Tango technology, a museum was augmented into a living rainforest. Walls were transformed into trees and corridors into forest paths and guests were given tablets which they could walk around and interact with endangered animals, something they can’t do in real life. The magic of AR is not just bringing virtual animals to your dining room table, but to your entire environment.

Or perhaps my favorite example, Google Earth VR. Imagine having the whole planet at your fingertips, one moment deftly flying through the skyscrapers of midtown Manhattan, and the next sitting peacefully at the top of Mount Everest. Taking data from satellite imagery and 3D photogrammetry, the environments in Google Earth VR are majestic, and are some of the most presence inducing in all of VR. There’s nothing more mind blowing than virtually standing outside of your apartment looking up at your window, knowing that you’re actually inside, decked out in VR gear. Furthermore, the experience really gets you excited about the VR’s future. If this is what it looks and feels like now, what will it look like 10 years from now?

The Future

But what about other VR experiences that could well and truly be useful for real life? In the automotive industry you could allow potential customers to drive through impossible test drives, while changing features and testing during the same experience.

Or, think about how VR can provide an amazing tool for training new professionals in technical fields.

What if you could train to be a wind turbine technician by running through a variety of possible scenarios before you ever step foot in one? Or, imagine learning a language. How useful would it be to simulate the feeling of language immersion by placing someone learning french in a Parisian cafe, and they can only navigate the experience by correctly pronouncing various phrases?

Or museums! Imagine going to the MOMA and seeing Jackson Pollock via an AR tablet ferverishly throwing paint at one of his canvases!

The possibilities are endless, but that doesn’t mean that every possibility is right for your brand. So before you spend a whole lot of your (or your client’s) money, ask yourself this. Why do you want to create a VR or AR experience? Do these technologies really provide your brand something better than other technologies? Or is your idea just a gimmick? If you’re looking for reach or engagement, then maybe wait a while before reaching for a VR headset. As the VR and AR markets mature, the channels for telling your brand stories will mature with them.

This article originally appeared on Shots on July 5, 2017.