The Evolution of Play in a Virtual World

I am speaking about “The Evolution of Play in a Virtual World” on Tuesday, at Kidscreen. I will be comparing play patterns using traditional toys and games, mobile games and AR/VR games. My basic premise is that AR/VR has some important advantages over previous generations of digital games and also includes some of the unique features that make traditional games so compelling.

Not to give away my entire talk, but here are some of the benefits of Virtual Play that I will be discussing:

  1. You can physically interact with realistic 3D characters, chasing them, etc.
  2. In Augmented Reality, the real world environment becomes part of the game.
  3. The types of interfaces available for interacting with the game tend to be direct and impactful, including touch, voice, and your whole body.
  4. It promotes a sense of presence as you move beyond 2D screens to 3D virtual worlds.
  5. Social interaction and sharing become possible in a realistic 3D space.
  6. Physical movement is an essential part of the game…no more couch potatoes!

Of course, so much has to do with how well a toy or game is designed, regardless of whether it is digital or physical in nature. Developmental psychologists value open-ended children’s products, so that the way they are used is limited only by the child’s imagination.  They value toys and games that engage a child’s interest and curiosity, without distracting them with an over abundance of bells and whistles. And they value products that have repeat playability, with sufficient variability and complexity to explore over time.

Girl playing with Playdough

My mantra as a parent was always the word “balance.” I wanted our four children to experience content on every available platform, because I felt that learning how to “decode” the medium was an important life skill.  It made no sense to me to take away TV or computer time; I simply limited it to 60 minutes a day. The key was finding content that enriched their minds, engaged their hearts, and promoted the values that were important to me.  Those kinds of kids’ toys and games you can find in every format, including new technologies.

The Top CES Technology for Kids

CES 2017 badge

CES is not the ideal place to showcase kids’ products. One hall is filled with car technology, another with smart health care devices, another with TVs, phones, etc. Even porn made a comeback this year, courtesy of virtual reality!

But if you’re persistent, you might find gold, as well as some products that are just so silly you can’t believe they made it to the convention floor. The one technology in particular that I think over time will have a big impact on kids’ products is artificial intelligence (AI) via voice activated assistants.

I wrote in a previous blog about our family’s love affair with Google Home and Amazon Echo. Not surprisingly, there are a couple of big companies trying to productize this technology for kids. It’s challenging, as you might expect. Children’s speech patterns are difficult for most smart speakers to decipher, and they make even more mistakes than usual trying to interpret their voices.  Have a 5-year-old ask Alexa, “Who is the 25th president?” and see what happens. It’s not pretty.

Despite this, I believe that voice assistants will become the primary interface for many children’s digital products. Every designer of kids’ products understands the difficulty of presenting choices to children, especially when text/reading is not a good option and you’re faced with a tiny screen. And having a device that is responsive to a child’s every question…all 200+ of them daily…seems promising to me.

Stepping into this minefield is Mattel, a company with history in the space and the gumption to take on the challenge. Mattel’s previous voice activated foray was with the Hello Barbie doll, followed by her Dreamhouse, both of which sparked almost instant controversy. My concerns lay less with privacy issues and more with the fact that my grandchildren, Jude (7) and Teddy (5) found the pre-scripted interactions with the doll frustrating and unsatisfying. It seemed that no matter what we wanted to talk about, Barbie would respond with an unrelated topic, e.g., careers or horses.

This summer Mattel is launching a speaker-based voice assistant called Aristotle, that functions like Google Home or Amazon Echo, but is designed for kids and parents. Unlike the Company’s previous products, Aristotle relies on the AI driven by Amazon’s Alexa and Microsoft’s Cortana. In addition, the device functionality changes as the child grows. Initially it can be a smart baby monitor, and respond appropriately to a crying baby with soothing sounds, as well as order diapers on Amazon. With toddlers, it will evolve into a teaching tool for colors, numbers, and the alphabet. For older kids, it will help with homework, teach a foreign language, and even introduce two-player voice games.

Aristotle by Mattel/Nabi

In addition to the smart speaker, Aristotle also includes a camera, which opens up lots of possible game and toy applications. Imagine using it with a special mat and Hot Wheels, so that any time the camera sees the fire engine leaves the fire station, you hear the siren. Or Melody, an American Girl doll, could be recognized by the camera and given a voice and personality through the speaker that reflects her 1960’s world. A child’s favorite book, e.g., The Hungry Caterpillar, could be read out loud and on command. Mattel plans to open up Aristotle to 3rd party developers, which will ensure a rich pipeline of applications.

Of course, all of this relies on Aristotle actually understanding what is being asked, a definite problem with kids. Mattel’s solution is to introduce voice training, through which the device learns the nuances of the child’s voice. That’s the theory; when the device ships this summer we’ll see how well it actually works. Another potential issue with these types of devices for kids, as mentioned previously, are privacy concerns. Aristotle, like Echo and Google Home, are on all the time, listening for a specific word, or cry. The information is captured on Aristotle’s servers, and forwarded to the parent’s phone. What happens if/when this information gets hacked? And how do we protect kids from the dark side of the internet and sites that we don’t want them to access?

Turns out most parents are not very concerned…yet.  According to this study, parents like the convenience of sensors and all different types of connected devices that make up the Internet of Things, e.g., baby health tracking devices, smart baby/children’s toys, smart home/home security devices, smart nursery/baby gear, and of course, smart speakers with virtual assistants. A third of the parents questioned even believe that these devices make them a better parent!

 From Fisher-Price's "The Future of Parenting" video

From Fisher-Price’s “The Future of Parenting” video

While I wouldn’t go that far, I am enthusiastic about the idea of kids having access to the same amazing wealth of information on the internet, without needing a smart phone.  And the curiosity of every child is something that should be carefully nurtured. Whether or not an artificially intelligent character who listens and responds to questions will provide that encouragement, only time will tell.

My Highly Idiosyncratic Picks for Holiday Gifts

It’s already late in the season for a list like this…even writing and reading this blog is a form of procrastination! But I have a few favorite gift tips that I wanted to share.  So without further ado.

For competitive kids who like to play games, but not together
How do you get a 7 year-old boy to play with two 5 year-old girls, and not cream them every time at a game? My granddaughters, Rosie and Teddy, were tired of losing, and refused to play with their cousin Jude anymore. Naturally, I couldn’t allow the girls to quit so I found, while searching on Amazon, a company I love, Peaceable Kingdom, and their line of cooperative board games. Starting at around 3-4 years of age, I recommend the game, Hoot Owl Hoot, and progressing to other games like Race to the Treasure for slightly older kids. Everyone wins or everyone loses. For the first time, I watched the children help each other figure out the best moves. Very gratifying for this grandmother!

For the business partner who likes a good story
I read about this book first on Bill Gates’ blog. I won’t repeat all the amazing reviews, just buy it. The fascinating, funny, inspiring “Shoe Dog: A Memoir by the Founder of Nike” by Phil Knight, will be appreciated.

For anyone who designs kids games
Mark Schlichting has been an admired designer since his earliest days at Broderbund. In addition to having credits on many classic kids games, he just wrote what I consider to be the best book available on the art and craft of designing children’s interactive products.  It is chock full of wisdom and warmth throughout the almost 400 beautifully illustrated pages. My favorite chapters include a brilliant discussion of traditional children’s play patterns, and how the best designers tap into those patterns in order to create something new. Other helpful sections discuss how to design challenges, leveling and pacing, all of which is extremely difficult to get right in kids products. But the best part of the book, in my opinion, is Mark’s discussion of gendered play patterns. It is a minefield, without a doubt, but he manages to put into words so many of the sex differences I have observed in children over the years, and how that relates to design. Friendship versus winning, action versus dialogue, epic story versus personal relevance, and more. Don’t miss it: Understanding Kids, Play, and Interactive Design.

For the man who has everything
My husband, David, has a new love in his life, “OK Google”. He talks to his new Google Home device each morning in dulcet tones, saying “please” and “thank you” as the voice activated know-it-all tells him the traffic, plays his favorite playlist and much more. At first Google Home retrieved his requested tunes on Google music, but after only a few times of specifying Spotify, Google grocked David’s preference. Machine learning is a beautiful thing and is probably why we will end up using Google Home more than our Amazon Echo, another voice activated smart assistant device.

If you want to be the coolest parent, or grandparent
Forbes just published this list of the best connected toys this Xmas. Connected toys combine the best of physical toys with added digital pizzazz and depth. I have always been a fan of connected toys because kids learn through physical manipulation — not everything can or should be digital. It’s the combination of physical with digital where the real magic happens. Of the products listed, I am a big fan of the Osmo products (although ridiculously pricey) and Maze by Seedling. Kids, ages 8+, build their maze with physical pieces, take a picture of it, then put on Google cardboard and walk through their maze in virtual reality. They can further customize and decorate their maze with obstacles, pictures, etc. Very cool, and although I am not a huge fan of VR for kids, short play periods with Google cardboard feels like a good introduction to the experience that all kids seem to crave.

If you need to butter-up a teacher
I recommend another connected toy that just started shipping this week, called Happy Atoms. You physically assemble models of molecules from atoms, which connect via magnets.  Then you scan the molecules you’ve built, and the app provides detailed information about its composition, name, properties, etc. Designed for ages 10-17, it combines the best of physical manipulation, discovery learning, and digital depth.

If you still aren’t convinced, maybe follow the advice of this article. Research tells us that the best gifts are the ones that people actually ask for!  Shocking, I know.

Happy Holidays!

Is Tango the Future of AR?

The much anticipated first Tango-enabled device is finally released to the public, along with Legacy’s new Augmented Reality game, Crayola Color Blaster. Where do we go from here?

promo_2For Legacy, the answer is easy. We have new content for Color Blaster releasing Valentine’s Day 2017. Instead of running away from zombies, you’ll have to avoid rampaging gnomes, orcs, and dragons. Seriously, we will be focusing most of our future efforts on evolving the types of interactivity possible with virtual characters, given our discovery of how much fun this is in AR.

unnamedWhat about other Tango apps? Phantogeist is another great game with roughly humanoid shaped aliens invading our world, along with monstrous creatures of different sizes! Both Woorld and Raise bring cute little characters and objects into your room. As the Tango technology becomes better and smarter, virtual characters will be able to recognize what objects are. Imagine virtual humanoids relaxing in your favorite armchair, digital pets sleeping in your bed, a 3D chef inspecting your kitchen and pots, and more!

hotwheelsAnother thing Tango devices do well is the ability to track your movement in space. This helps construction games in particular. Placing straight tracks, curves, accelerators, and much more is intuitive in the Hot Wheels Track Builder because you can pick up the objects and walk it over to exactly where you need it. The Towers for Tango game lets you build skyscrapers on your coffee table. Move your device up and down each floor and get an X-ray view to customize things exactly the way you want.

So back to the original question…Are Tango-enabled smart phones the future of AR? Or is looking through a phone as you walk through a room or building just too wonky to become mass market? Pokemon Go would indicate otherwise. There will certainly be more Tango smart phones to hit the market in the next year, including some announced at CES in January. (Just google it!)

On the other hand, even if Tango-enabled smart phones don’t explode in popularity, it doesn’t take insider information (of which I have none) to predict that Tango software may become embedded in Google Daydream VR headsets. Once that happens, Daydream will become a mixed reality platform (it currently just supports VR), and its form factor will just keep getting smaller and more comfortable.

My point is that I believe Tango software and 3D camera technology is here to stay. Its 3D mapping usefulness, fun factor, and relatively low cost will make it ubiquitous, eventually. The form it takes, however, will vary, at least in the short term. Mobile phones now…headsets next…glasses later? What do you think?

screen-shot-2016-09-24-at-09-39-06-796x364

6 Research Questions for AR/VR

We know surprisingly little about the long term effects of Augmented and Virtual Reality, on both kids and adults. The technology, thanks to large investments of resources and talent, is proceeding faster than our understanding of how to harness it or its consequences. Software developers and hardware manufacturers desperately need the insights obtainable only through verifiable research, before establishing best practices and policies.

chicago-toy-fair

Don’t get me wrong. I am an unabashed technology enthusiast. I believe that AR/VR offer the most exciting new business opportunity since the invention of smart phones. Legacy just completed our first major AR project, built for new “world-sensing” Tango devices. Yet producing Crayola Color Blaster on this remarkable new mobile platform has raised many cognitive and developmental questions for our team. Unfortunately, there is a dearth of reliable, valid research to guide our many design questions.  For example:

  1. How does a VR experience affect, if at all, the child’s developing brain and vision? What is the long term neurological impact of the two small LCD monitors and lenses, each projected at one eye, creating a stereoscopic effect and giving users the illusion of depth? Some hardware manufacturers have established age recommendations, but we already know that children are intensely interested in VR and will be using the devices regardless.making-immersive-virtual-reality-possible-in-mobile-27-638createstereoscopicimages
  2. In a world with unlimited “virtual garbage” potential, how do we direct the user’s attention towards relevant data and away from irrelevant stimuli? What does cognitive load research tell us about how to manage limited attentional resources? This may be our toughest design challenge. 571534199
  3. What are the societal implications of a world that has been customized to reflect our interests? What if our experience of the world is personalized through AR glasses, much as our Facebook feed is currently? Will this just further our descent into silos of interests and beliefs?
  4. How can we create more authentic digital tests of student or employee ability using AR/VR? What if, instead of problem solving tasks delivered via a computer or tablet, we embed virtual tasks in the real world instead? Can we build assessment tools that truly capture someone’s range of talents?
  5. Does memory work the same way for virtual objects as it does for real ones? In our AR game, Crayola Color Blaster, children seemed to have a hard time remembering the placement of virtual items in the room. If memory for virtual objects is worse than for physical objects, we will have to add more spatial cues to AR applications and design around that deficit.
  6. Can we operationalize a “sense of presence,” which many people believe results from an immersive VR experience? Are there corollary physiological measures? Does it lead to increased empathy, as some researchers maintain? Does this also imply that violent, FPS VR apps will have an outsized impact on our emotions?

I was a graduate student 30 years ago in Cognitive Psychology. Almost all of the other students in my program were preparing for a job in academia, the traditional next step. I was more interested in the application of theory, and ended up starting my own company to produce games and apps. Perhaps I should have stayed in academia long enough to study some of these important and fascinating research topics! The software development community needs answers sooner rather than later, given this technology’s vast potential for positive, or negative, impact.virtual-reality-classroom

A Family Affair – Creating the Trailer for Crayola Color Blaster

I love the trailer for our new Tango game, Crayola Color Blaster. While it doesn’t actually explain how the game works, it communicates a couple of things really clearly – it takes place in your real environment, you have to move around…a lot, and most importantly, it’s fun to play.

When you are a small company like Legacy, you make use of all the resources you have at your disposal. I am happy to report that it was all hands on deck in the creation of Crayola Color Blaster’s trailer.

  • Adam McClelland, daughter Rachel’s long-time boyfriend and talented filmmaker, shot and edited the trailer, dragging his unwieldy and expensive camera equipment to LA from NY.
  • Most of the gameplay footage was shot in my home in Los Angeles. (I should have cleaned up a bit beforehand!)
  • The trailer stars two of my grandchildren, Jude (7) and Teddy (5), who are the first two kids you see. The big kid, Leah, is my grown up daughter who likes to play games like her momma.
  • Our local recreation center, at Highland Park, was very supportive and allowed us to test and shoot 15 kids. Some of the kids are from there. Their gameplay feedback was incredibly important, especially on complicated interface issues.

Anyway, like everything else with this project, creating the trailer was really fun. Thank you everyone!

A Career Highpoint – Crayola Color Blaster Launches Today!

untitled-1When you’ve been doing the same thing for as long as I have, 30+ years, you have many career highs and lows. For example:

Highs:  Emergency Room, Law & Order, Murder, She Wrote, Crayola DJ games

Lows: Great Recession/losing our credit line, missing mobile, Twilight Zone

As you might imagine, each of those examples have stories attached, some of them painful. My point is that it is unrealistic to expect one’s career to just go up; my personal trajectory has been anything but.

That’s what makes today so sweet. Legacy’s latest game, Crayola Color Blaster, is launching in the Google Play Store. It was developed as part of the Google App Incubator Project; I can’t say enough good things about the support we received from Google. The app is specifically designed for Tango-enabled Android smart phones, like this Phab 2 Pro from Lenovo.

rocker

Designing a game for a new hardware platform, with innovative 3D depth sensing technology, is incredibly challenging. The interface, i.e., a “magic window” into the world, is completely unknown to consumers. Even more daunting, we had no idea what was actually fun. The game design, originally conceived of as a coloring book in the real world, morphed into a zombie color blaster fairly quickly when we realized that people loved interacting with virtual characters.

scientist

We had an amazing team for the task. My role on the project was as Executive Producer. As such, I assembled the team, managed the relationship with Crayola (very supportive and helpful folks), and gave many “notes” to the production staff. Legacy’s wonderfully creative producer and designer, Andrew Duncan, combined with the rather miraculous engineering of HitPoint Studios and artistic capabilities of Sixth Gear Studios, all came together for a wild six months of constant experimentation and failures, with a few striking successes, culminating in our launch today.

screenshot_20161012-171114

How lucky am I, to be able to do what I love for so many years, and then to experience this kind of career high? It’s the career arc that everyone dreams of, and energizes me to go out there and find the next challenging project. Any suggestions?

 

Polish, Polish, Polish

Many years ago, I heard Doug Carlston, CEO of Broderbund, talk about his approach to product development. I listened avidly, given that Doug was a much admired leader of the leading PC consumer software company at the time, publisher of Carmen Sandiego, Myst, Kid Pix, Printshop, and many other products that I loved.

What really struck me was Doug’s emphasis on the last 10% of product development, the “polish” phase. “It’s really hard. By this time your team is exhausted, everyone just wants to get the product finished and out the door and get their life back.”  But, he went on to say, “This is exactly the time that everyone’s energy and commitment is most critical. Just when the team has the least to give, that’s when they need to give the most.”

promo_1I often think about what Doug said, especially now that we are in the polish phase with Crayola Color Blaster. This is Legacy’s first augmented reality game, to launch at the same time as the Lenovo Phab 2 Pro, the first Google Tango device to enter the market. We just passed our beta milestone, and have about 6 weeks to finish a game that I have fallen in love with. What’s left to do? Feels like everything, examples below.

  • Voice overs and sound – Need to balance sound levels, adjust timing and conditions of voice input, add new music at dramatic moments, etc.
  • Beginning and ending of the game – Need title sequences, credits, celebration at game conclusion, promotion of next game chapter (coming 2017)
  • Disaster recovery/help – It’s new technology, and predictably, doesn’t work every time. What is the user experience when that happens?

All of these and more “polish” items are underway at the same time that our game is going through massive quality assurance and useability testing. Crayola Color Blaster is designed for ages 6+ and designing any app for children, much rather one for a technology that hasn’t even been released yet, has special challenges. Here are just a few of the things we have learned so far in testing.

  • The specific Tango device we are designing for is a cross between a phone and tablet. The 3D and infrared cameras are located on the upper back. We originally designed our game for landscape mode, but discovered quickly that many kids inadvertently cover the cameras with their fingers when tapping the screen to blast the zombies with color. We had to redesign the interface and gameplay to adapt to however the child chooses to hold the phablet in their hands, either landscape OR portrait.

lenovo-phab2-pro-phone

  • One of my favorite features of Crayola Color Blaster is that it is a “walk around” game. You are constantly moving, trying to stay away from the zombies so they don’t “crunch” your color and, if they do, finding paint buckets to suck up new color. This all works well assuming there is at least a 10’ x 10’ clear space in which to spawn new virtual zombies and ambient 3D graphics like flowers and rocks. But what about other environments without the requisite space? We had to design an “Endless Arcade” mode, in which the zombies come toward you from every direction, and you simply need to turn in a circle in order to color blast them.

Quality assurance in general is considerably more complicated with an augmented reality style game, given that it needs to play correctly even as it plays differently, depending on the physical environment. We’ve played the game everywhere we can think of, from elevators (my favorite) to basements to closets. Even so, I’m confident that many enterprising 8 year olds will foil our best testers, see below.

20160825_172321

It must be obvious by now that my wonderful team members and I will be very busy for the next few weeks, trying to add the level of polish that our first Tango game needs and deserves. I see many late nights ahead of us. Wish us luck!

20160825_173125

Crayola Color Blaster at Google’s Indie Game Fest!

Google nominated our AR game, Crayola Color Blaster, for their first annual Indie Game Festival, this Saturday, in SF.  There are two rounds of voting, with the final 5 games winning awesome prizes.

Among the 30 contestants, Legacy has the only game for kids (currently in beta), using a technology that’s not out there yet, for a hardware device that hasn’t shipped. Hmmm…

Wish us luck!!

Interacting With Virtual Characters in AR, and Loving It!

scary-zombie-imgAs a cognitive psychologist as well as game designer, I am intrigued with how traditional play patterns are evolving, given digital games like Pokemon Go that take place in the real world. Add sophisticated augmented reality to the mix, where virtual characters know where they are located vis-à-vis everything else, and it’s a game changer.

Think about it. What if, instead of simply collecting a Pokemon character, you could play tag with Pikachu? What if Haunter actually hid behind a rock and shouted “boo” as you walked by?  What if you were in a race with Growlithe to see who could get to the next Pokemon gym first? Given these kinds of real world, intelligent interactions, Pokemon characters would be dramatically more engaging than their 2D, screen-bound, counterparts. When virtual characters play along with us, we can reimagine almost every traditional game and gameplay pattern, from football to board games to dramatic play.

pokemon-go-google2
Charizard hides behind a real life rock in Google’s original April Fool’s video, a feature not present in the final game.

Why haven’t we seen this type of gameplay yet? A simple reason. The technology that supports it isn’t out in the marketplace yet. Before a 3D virtual character can interact with a game player, the avatar must know where it is in space. This kind of character “intelligence” requires 3D cameras and software that can scan an environment and learn the location of everything, including the game player. (A virtual Pokemon character can’t actually sit on a real park bench until it knows that an object with a particular shape exists in the real world.) Not until the Lenovo Phab 2 Pro launches this Fall, will there be a mass market phone that has the requisite Tango software and 3D cameras that can provide sophisticated augmented reality.

My company, Legacy Interactive, is developing an AR game for devices with Tango.  Since our licensing partner is Crayola and our app is geared to ages 6 and up, we originally thought about designing an AR coloring book. It was fun to color virtual 3D objects like trees and flowers appearing in actual room locations, but it quickly became obvious that this only scratched the surface of what is possible.

20160825_174928We thought about variations of games like tag, freeze tag, capture the flag, hide and go seek, scavenger hunts, etc. and how our coloring book app could be more interactive with the addition of virtual characters. The story line practically wrote itself! A mad scientist takes away all the color in the world, and sets wave after wave of colorless 3D zombies after you. Your task is to color blast them first, before they “tag” you and “crunch” your color, forcing you to find new color buckets and replenish your color.  Meanwhile, the zany professor taunts you between each wave; you have to ultimately find and chase him in the big boss fight.  All of this takes place inside an actual room, while you run around frantically, trying to avoid marauding zombies, and find new paint and special objects to appease them.

Screenshot_20160902-163317What are some other ideas for interacting with virtual characters in our upcoming Tango app?

  • The virtual character wants an object that is hidden in the real world. You must use logic to figure out where it is located and bring it to him.
  • Like color resist, you have to color blast your entire environment in order to “reveal” the virtual objects and “hidden” characters that have been there all the time.
  • Coloring special virtual objects placed in the environment, perhaps combined with a specific color, e.g., rainbow, sparkles, unlocks a special power up,
  • Change your room into new game environments by walking through a magical door or completing a puzzle.  “Poof!” We’ve re-skinned your bedroom with a whole new look and new gameplay parameters, allowing games to progress through a series of scenes and characters. Your room first appears as a dungeon, where you battle orcs, then a spaceship where you befriend little green aliens (my personal favorite).

screenshot_20160902-170508It’s hard to describe, until you’ve actually experienced it, how much fun it is to run around a room, trying to “tag” a virtual character before they tag you. And the more intelligent they seem, by changing their behavior according to where you are in the room or what physical objects are present, the more satisfying the interactions. With this first iteration of Tango, the device only knows that there is an object present of specific dimensions. Eventually the software will be able to identify what the object actually is, e.g. a ball, a table, a refrigerator, and how it can be used. This will open up even more types of interesting interactions with virtual characters.

I can’t wait. This is an extraordinarily exciting time to be designing games. The combination of real world locations, physical movement, and virtual 3D objects and characters that know where you are, provide a rich tapestry of opportunity for game designers.

Look for Legacy’s new AR game, Crayola Color Blaster, in the Google Play store this Halloween!