Retention has always been a challenge for both voice applications and chatbots: without dedicated space on a home screen or in an app drawer, it can be tough for users to serendipitously open a voice action (or recall that it even exists) unless they’ve formed a habit of using it.

In developing our first game with Google for the Google Nest Hub, Cookie Detective, the MediaMonks Labs team drew on its past experience of building games and in developing Google Assistant actions. The resulting game offers a nice benchmark for brands on how design elements and gamified features can come together and entice users to continue engaging with voice well into the long-term.

Long-Term Play Requires a Commitment to Long-Term Development

To be successful in voice, brands must understand that voice applications (and by extension, chatbots) aren’t a “set and forget” touchpoint. Just like with any interactive platform, you must commit to supporting a voice action into the long term, revising responses and the flow of conversation based on usage—for example, identifying which steps in the conversation users are likely to drop off and not come back.

“Your initial intuition during the creative process isn’t always right,” says Geert Eichhorn, Innovation Director at MediaMonks. “You must be willing to look at the data and see how people are using the voice action, or how they want to use it, and provide that for them. In the end, this leads to a better app and happier users.”

Realizing this, the mobile “games as a service” model—which has essentially boiled down FOMO into an art form—serves as a useful blueprint for how brands can achieve similar success in building loyalty through incremental updates. These games use a variety of fresh content, limited-time events, competitions and more to extend the shelf life of a game and keep players entertained over time. And with increased popularity of voice-enabled devices with screens like the Google Nest Hub or Amazon’s Echo Show, new possibilities will open up for brands to create more engaging and playful voice experiences.

Building a Compelling Voice Experience

Cookie Detective is focused on providing fun for young players, offering an appetizing, voice-based take on hide-and-seek, 20 Questions and Guess Who. It invites kids to ask ten questions to locate a hidden cookie within three virtual kitchens at varying difficulty.  The Labs team approached this project by combining its experience as voice developers with its wealth of knowledge in gaming accumulated over the years—especially in mobile and web game development, which both share similarities with the Google Assistant platform.

The premise of searching for cookies in the kitchen is based on the fact that voice devices are often placed in common areas where families gather. We focused on one of the most-trafficked places in the home (the kitchen) to ensure the experience fits the overall context of when and where players will engage with the game—perhaps when they’re on the hunt for a sweet treat IRL.

“We wanted this game to be something a kid could do while mum is cooking—no supervision needed, but with them together in the room,” says Eichhorn. “A cool addition to the game is the option for the parent to hide the cookie in the virtual kitchen using the touch screen (so the child wouldn’t hear the hiding spot–this is the only touch-only feature in the game).”

Fitting your voice action within a daily ritual like this can make it more habit-forming for the user. “This brings a really nice parent-child dynamic into the kitchen during an everyday activity like cooking and allows a potential reward to be earned for the kid as well,” says Eichhorn.

It’s also worth noting that the voice medium is unique for mimicking conversation on a human level. Therefore, the team found it was important to center the game’s narrative and gameplay prompts on its characters, whose lively voices reveal their personalities and bring their stories to life. In this way, dialogue (and its delivery) is not only fun and memorable, but functional.

And that tip doesn’t just apply to sound; if your game is played on a device with a screen, art direction and fluid animation also add a lot to the experience. If your brand has an especially strong mascot or IP that it can leverage through these elements, it’s easy to see how a brand can use voice to forge a strong relationship with players as they engage over time.

Enhance Loyalty Through Gameplay

In addition to some of the narrative and conceptual tips above about how a voice game can uniquely activate loyalty and retention, the team recommends that developers shift the approach from a one-off experience to an everlasting one where players can establish stronger connections with the game through added gamified elements.

One way to do this is to build a true sense of progression through unlocking items and rewards based on in-game performance or achieving milestones. This gives players a goal to aspire to and work toward. As they achieve new milestones, consider expanding the universe and story of the game to keep users interested or establish new habits.

This could be especially useful for brands related to health: imagine an assistant action that encourages children to brush their teeth twice a day without missing a day, perhaps using music and animations to ensure they brush for the correct amount of time. Likewise, a food brand might challenge members of the household to consume a well-rounded diet throughout the day.

Competition is another powerful motivator for encouraging replay value and loyalty. A leaderboard that refreshes on a timely basis (weekly or monthly, for example) is a simple way to achieve this. Notifications that players have just lost their leaderboard position can also get users back into the game. If your game is family-oriented, look for opportunities to pit family members against one another for some friendly competition!

Whether through lively characterization and narratives, brands can enable a closer relationship with consumers through voice—one strong enough to continue engaging over time. The competitive element that is unique to gaming, as well as frequent content updates that have worked so well for the mobile industry, can further encourage users to regularly check in and avoid churn. By combining these strategies together, voice becomes an effective channel for brands to build long-term relationships with consumers and fans.

See what else Labs has been up to.

For years, we at MediaMonks have racked up high scores in developing games with our clients, dating  back to the days of Flash. Today, we work across a breadth of platforms to create compelling  games that push the limits of technology—and most recently, we’ve worked with Google to take voice gaming to the next level with two new games for smart displays. 

Cookie Detective is an appetizing, voice-based take on hide-and-seek. It invites kids to ask ten questions to locate a hidden cookie within three virtual kitchens at varying difficulty. An upcoming game we’re working on is Junior Chef, which continues the culinary fun by challenging kids to instruct a team of chefs in their own  restaurants.

Both games aim to encourage play-based learning (children learn about international ingredients in Junior Chef, for example). As an Easter egg, the game also welcome positive behaviors, such as speaking politely to the chefs.

As the games were developed for the Google Nest Hub and Hub Max, we aimed to bring together the best of screen and voice-based interactivity. We relied on the Interactive Canvas framework to power-up even more possibilities for voice technology and the role it plays in the home.

What We’ve Learned 

Look Ma, No Hands! 

Historically, players have interacted with games using keyboards, mice, controllers and on screen buttons. While traditional controls are ingrained in the medium, voice offers a couple of new challenges: developers must account for a variety of variables like accents, context, subjects, phrasing and more. These interactions must be as accessible and intuitive as talking to another person. 

Context is Crucial 

Voice devices are often placed where the family gets together. We kept this in mind when developing our games, focusing on one of the most-trafficked places in the home: the kitchen. We  chose activities natural to the kitchen (like cooking, learning recipes and looking for cookies) for the core ingredients of our two games. 

Build on Previous Experience 

Google offers several tools for building Actions, most notably Actions Builder, which lowers the barrier of entry by eliminating the need for technical development  to create simple Actions. For our games, we combined the Actions SDK, the Actions on Google client library, the Interactive  Canvas library and our solid background of web development to build a cohesive framework and robust architecture for building voice-enabled games. 

We approached this project by combining our experience as voice developers with the wealth of knowledge in gaming that we’ve accumulated over the years—especially in mobile and web game development, which shares similarities with this platform. With Cookie Detective and Junior Chef, we aim to provide a benchmark for what’s possible, discover what works best, and learn how we can push boundaries even more in the future. 

Let Characters Drive Stories & Interaction 

Voice-based gaming offers an opportunity to deliver a rich story featuring characters with distinct personalities. While we made ample use of text and animations in our games, we relied on our characters’ voices to reveal their personalities and bring their stories to life. Expressions are not only fun, but functional: if a command isn’t clear in Cookie Detective, the cookie will respond based on the situation, for example the cookie could say, “I didn’t catch that, you can ask me something like, are you in the cookie jar?”  

These responses can help immerse players into the game, though we found that it’s important to balance dialogue that serves characters’ personalities with lines that support gameplay. Deliver something playful the first time a player makes a particular action, followed by something short and concise for recurring interactions. So, when our cookie friend once again can’t understand what a player has said, he will still give a response that fits the situation, for example: “You can ask me something like, is it green?” 

Keep it Fun – and Fluid 

Keep in mind that a game made for the family should be about having fun, even when you lose. If players end up making a less-than-stellar meal in Junior Chef, for example, they’ll still take delight in watching how their guests react with exaggerated, funny faces. Whether it’s someone’s first time playing the game or their first time using voice, this eases pressure felt by the player and encourages them to come back and try again. 

And when it comes to animation, we realize that smart displays aren’t trying to be in the same category as home gaming consoles. With performance and a casual audience in mind, stick to simple, colorful vector graphics and choose when to trigger animations wisely. We used Bodymovin for both games, which gave good results despite being a bit limiting. 

Design Responsibly 

When creating games for children, the potential for an emotional connection with the voice assistant is significant: studies have shown that interactions between kids and voice assistants reinforce patterns of behavior. We considered these responsibilities heavily, as well as others, including privacy and addictive screen habits, by designing each game to encourage polite offline social interactions.

Taking Voice to the Next Level 

In addition to the learnings above that we’ve incorporated into the development of Cookie Detective and Junior Chef, we realize there are more opportunities to take voice-based gaming to the next level and increase user retention.

Game developers can keep users coming back by offering: 

1. Unlockable items that enhance gameplay and reward player progress
2. Long-term support that adds new levels or expands the story 
3. Competitive elements, like daily challenges and time-based leaderboards 
4. Notifications that alert users to new content or a lost position on the leaderboard
5. Improvements based on tracking user behavior 

We love how this experience allowed us to bring new elements to gaming and recontextualize the role they play in the home. We also realize that we’re in the early days for these kinds of gaming experiences, and look forward to future possibilities like support for endless voices, supporting real time audio effects or cadence, richer voice processing, and even stronger AI.  

One thing we’re sure about the state of voice gaming today is that the interaction between people and devices is more human than ever. Based on our experiments in building games  for the Google Nest Hub devices, we believe Google Assistant will undoubtedly play a key role in driving more human-centered interaction and fulfilling the never-ending need for play.

At this year’s I/O conference, Google unveiled several new features related to its upcoming Android release and devices. Among the most exciting of these features are those that aim to change the dominant interface through which users engage with their devices: typing on a keyboard. While the touch screen revolutionized media about a decade ago, it looks like the camera and microphone are ready to take the baton, at least when it comes to accessing on-the-go info.

Google CEO Sundar Pichai’s keynote event was brimming with new features and products that are set to change how we interact with devices and each other in everyday situations. We paid tribute to this spirit of progress by producing an animated countdown video that kicked off the keynote, taking viewers on a journey through advances in tech over the decades.

Many of Google’s most interesting announcements centered on voice and visual search in particular, making the case that these new features could provide its users with information much faster than if they had to type it out. While this has always been the idea with voice interfaces, this year’s I/O event delivers on the promise by showcasing instantaneous voice recognition with its Assistant and surfacing up actionable information through augmented reality. Here are the feature’s we’re most excited about—and where the value lies for users and brands alike.

Google Assistant Becomes More Human

Google announced several improvements to its Assistant, many of which center around on-device voice recognition. Previously, Google’s voice recognition model was 100GB, requiring queries to connect to the cloud before getting a response. Now, Google has managed to shrink that model down to an impressive half-gigabyte, small enough to fit on devices for rapid, offline voice recognition.

“Now that Google can do recognition on the device itself, the device can actively listen and respond without the need to go through the loop of saying ‘Hey Google’ followed by a command,” says Michiel Brinkers, Technical Director at MediaMonks. “You can simply keep talking to the Assistant with follow-up questions.” It also allows for contextual commands, such as saying “Stop” to stop media playback on the device, no “Ok Google” required.

Google’s newly unveiled Nest Hub Max device—which is a mix between the Home Hub and Nest camera—even adds physical gestures to the interface. Thanks to facial recognition, the Nest Hub Max can alert users when it notices someone in the home it doesn’t recognize, or greet them with personalized content when they’re in view. The latter solves a crucial problem faced by Internet of Things devices: when multiple users in a home share a single device, how do you target them individually with personalized content? We’re excited to see Google crack the case while alleviating privacy concerns with on-device facial recognition.

Speedy Voice Recognition Will Change Users’ Lives

The greatly improved speed achieved through offline Assistant interactions is a game-changer on mobile devices, where wait times or lack of connection can be a huge pain point. “If a voice assistant doesn’t instantly do what you want it to, or if it gets it wrong, then it becomes more effort to use that system than to accomplish the task through typing or tapping,” says Brinkers. “But what Google showed offers a huge improvement.”

While on-device voice recognition can make many of our lives easier, but for some it will be life-changing: thanks to immediate transcriptions, Android devices will be able to provide users with auto-generated subtitles for any video or audio (including live content), an obvious benefit to the hard-of-hearing. In addition, Google announced its Project Euphonia program, which will provide larger data sets to train the Assistant to better understand those with speech impairments. “These initiatives are where a company like Google shows their value to the greater good,” says Brinkers.

As a Technical Director, the faster, improved speech recognition turns the creative wheels in Brinkers’ mind. “If voice does become a dominant input method, maybe we can listen to tone of voice—do more than just listen to what’s being said, but how it’s said,” he muses. “Then we could identify their emotion and design experiences around that.”

Google Lens Brings Printed & Digital Content Together

Voice isn’t the only interface Google is gunning for this year: the company also revealed several new AR features. While most consumers’ experience with AR has been focused explicitly on entertainment, I/O demonstrated how much the technology has matured in the past year to provide users with actionable, contextual information they can use in their daily lives.

One example shown in Google’s keynote is the ability to scan a restaurant menu with a phone using Google Lens. Doing so provides users with a list of a restaurant’s most popular dishes, reviews and photos. We’ve long said that the camera is the new browser, and new Lens features offer a textbook example of what that future could truly look like. “If I could read any restaurant menu in a foreign country and see what the food looks like through my phone, that would be amazing,” says Brinkers. “We always ask ourselves what the utility use case is for AR. This is it.”

In addition to providing greater contextual information, Google showcased Lens’ ability to animate traditional, static media—one of the coolest features for those who always wished they could read an animated newspaper as seen in the Harry Potter universe. One example demoed at the event is a poster depicting the Eiffel Tower. When scanned with Google Lens, clouds in the sky begin to move, bringing the image to life.

The tech isn’t just about cool visual effects, though—it also has utility, particularly with how-to content. Scan a recipe in a magazine with Lens, and a video tutorial can overlay atop it to show how the dish is prepared. What really places Lens at the forefront of AR is that the scanned media doesn’t require abstract, distracting markers or QR codes to activate; the content itself is the key, enabling a more elegant way to augment printed media.

Get Up-Close & Personal with Google Search Results Using AR

Later this year, users will find 3D models in Google search results, allowing them to examine the object or thing they’re searching for from any angle. If that’s not already cool enough, Google is upping the ante by letting users place the object in front of them using AR. This functionality offers a simple, intuitive way for users to learn about real-world objects and preview products.

“If you searched a chair on Google, it would be neat to drop it down in your room and see how it looks,” says Brinkers. “It will be interesting to see how this competes with proprietary apps that already let you do something similar.” One benefit that searchable AR objects have over those native apps is that users can view them without having to download and install anything. Google is exploring brand partnerships for developing these models in search, signaling the potential value it can have for marketing.

What’s truly exciting about each of these developments is their potential to come together in one unified experience. Scan a sign in a foreign language with Lens, for example, and Google can verbally read it back to you in your own language through advanced text-to-speech. Marry visual and voice features with an augmented reality layer, and the way we interact with everyday devices—if not the environments around us—may radically alter in the next couple of years. “What’s interesting with this event is that you see a lot of synergy between AR and machine learning,” says Brinkers. “Google is combining all these tools that they’ve worked on separately, and we see it coming together in a way that no one anticipated.” Ok Google, what’ll it be next?