Crayola Color Blaster at Google’s Indie Game Fest!

Google nominated our AR game, Crayola Color Blaster, for their first annual Indie Game Festival, this Saturday, in SF.  There are two rounds of voting, with the final 5 games winning awesome prizes.

Among the 30 contestants, Legacy has the only game for kids (currently in beta), using a technology that’s not out there yet, for a hardware device that hasn’t shipped. Hmmm…

Wish us luck!!

Interacting With Virtual Characters in AR, and Loving It!

scary-zombie-imgAs a cognitive psychologist as well as game designer, I am intrigued with how traditional play patterns are evolving, given digital games like Pokemon Go that take place in the real world. Add sophisticated augmented reality to the mix, where virtual characters know where they are located vis-à-vis everything else, and it’s a game changer.

Think about it. What if, instead of simply collecting a Pokemon character, you could play tag with Pikachu? What if Haunter actually hid behind a rock and shouted “boo” as you walked by?  What if you were in a race with Growlithe to see who could get to the next Pokemon gym first? Given these kinds of real world, intelligent interactions, Pokemon characters would be dramatically more engaging than their 2D, screen-bound, counterparts. When virtual characters play along with us, we can reimagine almost every traditional game and gameplay pattern, from football to board games to dramatic play.

pokemon-go-google2
Charizard hides behind a real life rock in Google’s original April Fool’s video, a feature not present in the final game.

Why haven’t we seen this type of gameplay yet? A simple reason. The technology that supports it isn’t out in the marketplace yet. Before a 3D virtual character can interact with a game player, the avatar must know where it is in space. This kind of character “intelligence” requires 3D cameras and software that can scan an environment and learn the location of everything, including the game player. (A virtual Pokemon character can’t actually sit on a real park bench until it knows that an object with a particular shape exists in the real world.) Not until the Lenovo Phab 2 Pro launches this Fall, will there be a mass market phone that has the requisite Tango software and 3D cameras that can provide sophisticated augmented reality.

My company, Legacy Interactive, is developing an AR game for devices with Tango.  Since our licensing partner is Crayola and our app is geared to ages 6 and up, we originally thought about designing an AR coloring book. It was fun to color virtual 3D objects like trees and flowers appearing in actual room locations, but it quickly became obvious that this only scratched the surface of what is possible.

20160825_174928We thought about variations of games like tag, freeze tag, capture the flag, hide and go seek, scavenger hunts, etc. and how our coloring book app could be more interactive with the addition of virtual characters. The story line practically wrote itself! A mad scientist takes away all the color in the world, and sets wave after wave of colorless 3D zombies after you. Your task is to color blast them first, before they “tag” you and “crunch” your color, forcing you to find new color buckets and replenish your color.  Meanwhile, the zany professor taunts you between each wave; you have to ultimately find and chase him in the big boss fight.  All of this takes place inside an actual room, while you run around frantically, trying to avoid marauding zombies, and find new paint and special objects to appease them.

Screenshot_20160902-163317What are some other ideas for interacting with virtual characters in our upcoming Tango app?

  • The virtual character wants an object that is hidden in the real world. You must use logic to figure out where it is located and bring it to him.
  • Like color resist, you have to color blast your entire environment in order to “reveal” the virtual objects and “hidden” characters that have been there all the time.
  • Coloring special virtual objects placed in the environment, perhaps combined with a specific color, e.g., rainbow, sparkles, unlocks a special power up,
  • Change your room into new game environments by walking through a magical door or completing a puzzle.  “Poof!” We’ve re-skinned your bedroom with a whole new look and new gameplay parameters, allowing games to progress through a series of scenes and characters. Your room first appears as a dungeon, where you battle orcs, then a spaceship where you befriend little green aliens (my personal favorite).

screenshot_20160902-170508It’s hard to describe, until you’ve actually experienced it, how much fun it is to run around a room, trying to “tag” a virtual character before they tag you. And the more intelligent they seem, by changing their behavior according to where you are in the room or what physical objects are present, the more satisfying the interactions. With this first iteration of Tango, the device only knows that there is an object present of specific dimensions. Eventually the software will be able to identify what the object actually is, e.g. a ball, a table, a refrigerator, and how it can be used. This will open up even more types of interesting interactions with virtual characters.

I can’t wait. This is an extraordinarily exciting time to be designing games. The combination of real world locations, physical movement, and virtual 3D objects and characters that know where you are, provide a rich tapestry of opportunity for game designers.

Look for Legacy’s new AR game, Crayola Color Blaster, in the Google Play store this Halloween!

 

Emerging Play Patterns With Augmented Reality – What Works and What Doesn’t

How does the nature of play change in a world with augmented reality, when your device knows where you are and everything around you?

Pokemon Go broke through gaming’s summer malaise in a spectacularly dramatic fashion, due to its revered brand, fun collector-style gameplay, and “gee whiz” location-based augmented reality integration. But one of the most interesting aspects of Pokemon Go is, in my opinion, how it provides a glimpse into unique play patterns that are enabled by new technologies. Where do we go from here?

The Play Observation Scale (POS) is a good general purpose framework for thinking about the types of play and how it changes over the course of child development: functional (characteristic of very young children), construction, exploration, dramatic, rough and tumble, and games with rules. Within this, play can be solitary, parallel, or group (with one common goal).

stateofplay

Let’s focus on construction play first. (I will address other play patterns in subsequent blog posts.) Traditionally, this refers to assembling blocks, sewing doll clothes, building a fort, and more. Any time a child is a “maker” with an objective, that is considered constructive play. What happens when a child builds something with virtual items? Given AR possibilities due to 3D powered technologies like Google Tango, we now have examples of virtual items actually interacting with physical objects to create an entirely new experience. For example, in Woorld, you can add a spigot to a table top, turn it on, and watch your room fill all the way up to the ceiling with virtual water. To empty the room, you simply add a virtual drain to the floor and watch the water level go down. So much fun!

app-woorld

Here’s an idea. What if there were an AR app where kids power their own virtual machine capable of creating magical virtual objects of any shape and size? Once the machine has been constructed to the player’s satisfaction, a simple touch of a button or lever shoots out a stream of… what? You won’t ever quite know until you try. What will come out the end of the machine will be related to the components that have been added and used previously. For example, let’s say you added a Chicken Button, a Time Machine Rewinder, a Color Randomizer and an Anti-Gravity Extruder, which results in…flying purple eggs! You can move, drop, throw, and scale all the virtual objects created, filling up your room to the ceiling.

thingamajig

This game idea sounds like a dream to someone like me, who for 30+ years has championed the design of products that support open-ended constructivist theories of learning.  Of course it would be even better if the child could create some of the components and not have to rely on just mixing what is in their digital catalog. That would be “Crazy Contraptions” on steroids!

What are the benefits of adding AR to construction play, and is there a significant difference between constructing with virtual objects compared to real ones? Digital items can be easily edited and changed, unlike physical objects. Size them up or down and place them anywhere…even on the ceiling! They are vastly more adaptable than physical objects and allow for creative combinations not possible with real objects.

The disadvantage of virtual objects, and it’s a big one, is that there is currently no haptic feedback with mobile AR devices like Google Tango and precious little with VR either. You don’t feel as if you are actually touching something; you don’t have the experience of twisting and manipulating the object. You can’t feel the screw loosen with the screwdriver, or feel the hammer’s impact on your arm. For children, who learn initially through direct sensorimotor experience with the world, this is a significant issue. The younger the child, the bigger the problem, although the lack of haptic feedback in AR/VR, especially when the experience is mechanical in nature, impedes learning at every age.

lego_75102_box1_na_1488

At this stage of the technology’s evolution, assembling your new Star Wars Lego X-Wing Fighter with real blocks, compared to virtual ones, would be easier and more satisfying. But the finished product couldn’t launch into hyperspace, lost in clouds of stars and planets, like it could in AR. Perhaps the answer is a combination of physical toy with AR? To be continued…

Augmented Reality Run Amok

As a game developer actively working on various AR platforms, I have been mesmerized by the success of Pokemon Go. (And so has everyone else in our business.) Given the number of Pokemon Go knock-offs already in development, there will be many more outdoor, GPS-powered augmented reality games launched soon.

pokemon-go-list1-1200x682

At a recent meeting with my colleagues at Legacy, we were discussing some of the implications of Pokemon style games for advertisers.  The market for virtual advertising, which is almost nonexistent now, will grow exponentially as more AR games and apps that allow you to superimpose virtual images on real objects, indoors and out, enter the market.

I started to riff about how everyone will sell the digital rights to their front yards for virtual billboards. Because of my home’s close proximity to the entrance of Griffith Park (the largest park in LA), will my property be worth more in the future due to all the foot traffic? Patrick, a Legacy employee who enjoys poking holes in my arguments as a matter of principle, quickly dispensed with my Gold Rush dreams.  He shared this dystopian trailer of the future, where EVERYTHING in our environment is polluted with “helpful” avatars and advertising, thanks to augmented reality. Tell me if you don’t feel like poking your eyes out after watching this short film. https://vimeo.com/166807261

571534199_1280x720

Then, to top it off, another Legacy employee, Andrew, reminded me of a famous science fiction short story, The Subliminal Man, by J.G. Ballard.  It is a scary depiction of a consumer-obsessed culture in which subliminal marketing messages are blasted nonstop at unsuspecting workers. I won’t spoil the horrifying ending, but I’ve already hinted at it.  https://biblioklept.org/2013/11/29/the-complete-short-stories-of-j-g-ballard-fifth-riff-the-subliminal-man/

How can we prevent this possible, utterly repellent future? It’s going to require years of discussion and negotiation between technology companies, advertisers, consumers, and government. Can we use hard-won lessons in water, air, noise pollution oversight and policy to avoid polluting our virtual environment? It’s up to us and time to start the conversation.

EMGNpollution2

Video Games Are Good For You!

IMG_3168Guest Blogger, Nicholas Maryan, is an awesome summer intern at Legacy Games.

I’ve played video games my entire life. My mom used to yell at me frequently for “wasting my time.” Yeah, I could have spent a few more hours a week studying my multiplication tables or actually reading that book for my report. Little did my mother know or appreciate that all that time I was working on other important skills.

As a result of playing games, my manual dexterity improved significantly, as well as my critical thinking, decision-making, and ability to type (imperative to efficiently writing papers and emails). In fact, as a result of playing World of Warcraft, I was top of my class in WPM as well as winner of my 5th grade spelling bee. Words like “pristine” and “ogre” were easy for me by that time. And years of playing adventure-based games enhanced my critical thinking skills, forcing me to constantly analyze where to go next (hint: always follow the enemies), strategize how to defeat bosses, or traverse difficult terrain. Games like Call of Duty, in which much of the time is spent playing against other live opponents, help develop quick decision-making skills by anticipating and reacting to others’ moves. Though the specific content (e.g., combat), may not be useful in the real world, the ability to process information quickly and with precision is a vital skill.

WoWphotoforblog
Research has shown that the act of playing video games can improve various skills the more people play. A few studies found that those who play video games develop improved visual attention strategies (i.e. multi-tasking) and are better at analyzing entire scenes and situations rather than one specific object.

Video games, particularly puzzle games, have been found to increase positive emotion, reduce anxiety, and promote overall relaxation. It goes beyond de-stressing as well. There are a slew of mobile games designed to exercise young children’s creativity, dexterity, vocabulary, math, and even cooking skills (just check out Toca Kitchen). These are benefits of the classic console, computer, and mobile games. But what about the new age of Augmented and Virtual Reality?

For those who are new to those terms, augmented and virtual reality are pretty similar. Virtual reality is a completely immersive experience (you have probably seen the headsets) which entirely takes over your visual and auditory senses. Augmented reality does exactly what it says it does, i.e., it augments reality. It imposes digital images onto the real world; the most pertinent example currently is Pokemon GO. Now you can catch “real life” Pokemon from the comfort of your own smartphone.

PokemonGoblogphoto
Since the franchise has been successful for twenty years, Nintendo has captured perhaps the largest age range of any other game in the market today (the nostalgic 30-&-up, down to young kids). The staggering usage numbers are great for Nintendo and the developer Niantic, but this revolutionary game also presents health benefits that could change the industry.

According to one study done by Oppezzo M. and Schwartz, D.L., (2014) participants who walked more, especially among nature, tended to be more creative in their flow of ideas. Another study done by Bratman, G.N. et al. (2015) looked into the effects of urbanization and mental health, and found that those who walk around in nature for an extended amount of time have less negative thoughts on average. Pairing that data with the ability to interact with a mobile video game can have endless positive effects, and learning opportunities. This is where Pokemon GO shines.

Of course, it has also encouraged some bad habits such as blindly walking around lost in your phone screen, resulting in folks falling in a pond, finding a dead body, and crashing cars. Of those who suffer from depression, some claim the app motivates them to get up and get out of the house. In fact, the other day a girl I went to high school with, who suffers from severe depression due to chronic pain, posted a Facebook status celebrating the fact that she finally had a reason to spend time outside.

This alone shows Pokemon’s true potential. No longer is AR seen as simply a more complex and innovative way to play games. It can be a new tool in fighting depression and other mental diseases silently undermining our society. Augmented reality done right can help encourage more interaction between the outside world and gamers of all types, including folks who suffer from mental illness.

As for video games in general, they aren’t all that bad. Video games can even be beneficial for children when used in moderation. I am excited to see what the future of video games holds. One thing I know for sure is that the future is bright, and full of pixels. Have I convinced you yet, Mom?

Nicholas Maryan is a Senior at Kenyon College and is studying Economics. He plans to graduate Spring of 2017. Hire him!

A Book About Love

00010045

Normally I write about kids and technology. Today’s topic concerns a single kid, my son Jonah, who just wrote his fourth science book, A Book About Love. David Brooks writes, in a review in the New York Times: “The book is interesting on nearly every page.”

My favorite chapters in A Book About Love are the ones on parent-child love and attachment theory, in which our earliest relationships are seen as the template for every subsequent close relationship. Jonah makes much of the fact that the kind of love that lasts a lifetime takes effort and grit. His own story exemplifies this. I couldn’t be more proud of Jonah and the beautiful book he has written, despite some stiff headwinds.

Again, from David Brooks: “He mixes a wide range of reference, both scientific and literary, in a way that is sometimes familiar but sometimes surprising and illuminating. Good writers make writing look easy, but what people like Lehrer do is not easy at all.”

Buy the book, read it, and enjoy!

IMG_2550

The Future of Digital Play…Maybe.

I’m reading an interesting book about the play patterns of rats, by Sergio and Vivien Pellis. It’s called The Playful Brain, and presents a strong evolutionary argument why play is critical, across species.  If rats don’t get a chance to “play/fight” with their peers, they are basically a hot mess. Play-deprived rats can’t even figure out how to make babies, and try to mount from the wrong side! “Play as a calibrating mechanism for emotions, motor control, stress reduction, role relationships…is strongly documented.”

IMG_3067

We can all agree that play is critical for normal development. How does this relate to the current state of children’s digital apps?  As someone who is in the business of creating kid’s apps, I have to admit to a certain amount of ambivalence towards our industry.  Apps are ideal for quiet time on the couch, when mom or dad is making dinner and as an alternative to TV time. They aren’t, however, social in nature. Apps are also sedentary, and we all know the problems with that, for both kids and adults. Another reason for my ambivalence is the crushing amount of mostly me-too apps that are available in the app stores. There are still good games coming out, but I am increasingly bored with the offerings, and definitely see a fatigue on the part of parents who are simply not downloading as many kids apps as they used to. This in turn discourages new app developers with fresh ideas, who see no path to profitability.

Play in the Future, with AR

I think it’s time we rethink play in the digital space. This week NPR had a great story about a new version of Pokemon that uses augmented reality, tied to notable landmarks in your environment, coming out in July. Looking through your smartphone, you see a wild Charmander  (virtual, obviously), sitting on the grass or floating in the air. You can train it to be yours. The game uses GPS technology and a large database of locations to blend gameplay with your real environment. This has been done a few times before, but never with a mass market, 20-year-old kids brand like Pokemon. 

 Which brings me to Legacy’s newest app for kids, AR Worlds (working title). We’re using a smartphone that isn’t available until this Fall together with Google Tango, so some of what I am about to describe will sound like science fiction. It is the first consumer device to support augmented reality plus wayfinding, room scanning and more, which it does by including specialized software and 2-3D cameras. 

AR Worlds is a combination arcade shooter and coloring book, in 3D, for kids ages 6+. Once your actual house, its walls and objects, are scanned by the Tango device, you move around the rooms, looking through your phone, for virtual objects to color. Creatures spawn randomly around you, moving towards you until you aim, fire, and color them. (I was playing hide and seek with a zombie the other day in our office, pretty great!).

Screenshot_2016-06-23-09-53-02 Once they are colored, some of the virtual characters need something else, e.g., the baseball zombie is looking for a bat. You search around your room until you find what they need, at which point they do a happy dance. Soon your bedroom is full of colorful virtual zombies, birds, dragons, and more, all come to life!

I love the old fashioned play patterns in our game…everything from hide and go seek, coloring, and searching for hidden objects…and how they can be enhanced with the addition of digital technology. Plus having a virtual character follow you as you move around the room is mind-blowing. Kids love to be the boss, and this is the ultimate in control.

I believe that AR/VR technology on the horizon is going to completely shake up our ideas of what is possible with digital play. And it’s past time for some new thinking on the subject.  Look for the new Lenovo Phab 2 Pro and Legacy’s new app this Fall in the Google Play Store.

E3 Impressions

IMG_3013

As anyone who has worked with Legacy knows, the real company talent lies in our amazing staff. So, I asked three of them – Adam, Andrew, and Patrick – to give me their impressions of last week’s wild and crazy Electronic Entertainment Expo (E3) in Los Angeles. While smaller than previous shows, it still provides an accurate reflection of the state of the video game industry (and also still induces brain seizures, given the level of flash and noise).  Enjoy!

zelda_e3_2016

TAKE 1

 Nintendo had an absolutely brilliant showing of Zelda: Breath of the Wild. The fact that it was the only major demo in their booth was a big departure for Nintendo and shows just how much faith and support they are pouring into this franchise. The gameplay appears to continue taking inspiration from large, open world experiences and introduces old and new toys and gizmos. And it does so with a very charming, very familiar gameplay and art style from Windwaker and Twilight Princess. Every viewpoint, every nook and cranny begs to be poke, prodded and explored—and that’s exactly what I want out of my Zelda. Gliding on the wind, climbing mountains, chopping trees, taming horses, frying up food, physics based gameplay. Sign me up! Hype Meter: 11/10.

Abzu-screenshot-1

TAKE 2

I go to E3 for those ineffable moments of experience and presence that you can’t get from watching a video. A few years back, it was Kinect and other motion-tracking software. Watching someone do it and experiencing it yourself are very different propositions. Now it’s VR.

But the most memorable experience for me was getting a chance to experience the long-awaited Abzû from developer Giant Squid. Created by a small studio headed by the art director of the award-winning Flower and Journey, it’s a beautiful (and I mean beautiful!) undersea exploration game. No blood, no combat. Just enchantment, relaxation and a gently mystical undersea world that transported me far away from the bedlam of the Convention Center (Releases on PS4 and Steam Aug 2nd)

playstationvr

Take 3

For me, the focus of E3 this year wasn’t so much on the games themselves, but on the technology that drives those games, specifically VR. Almost every major company had something to show in the VR space, whether it was adding a VR experience to existing games like Skyrim and Doom, bringing old franchises to the cutting edge like Resident Evil 7, or creating brand new IPs strictly for VR like Raw Data by Survios.  

Adam, Andrew, and Patrick hit on the highlights, thanks guys. For my part, I wish there were fewer 1st person shooters and sequels at E3. On the other hand, there were definitely fewer booth babes than in years past! Maybe some day our industry will truly reflect the diversity of game genres and customers. One can hope.

SLJ1505-3D-Lib-Students

3D Printing in Schools – A Fad or Here to Stay?

It’s back to the future, all over again. The trend in schools to create Maker Labs, complete with 3D printers, laser cutters, etc., is exciting, but really not that different than traditional shop classes. Of course, modern tools are now infused with technology, but there’s another difference that may be even more critical to the success of Maker Labs: software.

In order to create something from scratch and then 3D print it, students must learn how to use a 3D modeling software program. These programs are difficult, to say the least. Even an entry level product like Tinkercad requires that you think in 3D, on a 2D screen. The ability to mentally rotate a 3D object varies widely from person to person. Plus, it takes practice to think about subtracting space from a 3D object (which is why sculpting and woodcarving are difficult). Finally, designing objects to be water-tight and with surfaces and support structures that allow for actual 3D printing is challenging.

3dmodelclass

In other words, it simply isn’t enough to teach students how to use a 3D printer. We also have to provide 3D modeling software that is easily mastered and additional educational software that directly supports curriculum objectives. Let’s face it, students love 3D printing, and that enthusiasm needs to be harnessed to achieve specific learning outcomes.

What if we create educational software that allows students to simulate different options for their 3D project, prior to printing it out? Through this process, students can test out their assumptions and learn from their mistakes, without wasting valuable filament and print time. Legacy Interactive (my company) is working on a 3D printed microscope that does exactly that.

Legacy’s unique simulation software allows students to make limited changes to a 3D model, and get accurate feedback, prior to printing. Students arrive at a correct solution, through a trial and error process informed by the relevant math. Here’s how it works:

  1. Students can vary the length of the microscope’s body tube; there are 3 different settings to choose from.
  2. Students can vary the placement of 4 different lenses, vary the orientation of the lenses (convex VS concave), and vary the distance between the lenses.  
  3. There is an optimum relationship between the length of the body tube and the placement and orientation of each lens, that is determined by algebraic equations provided in the student worksheets. Included in the teacher materials is information about the science of optics, convergent and divergent lenses, magnification, focal length, etc.
  4. Students try out different solutions and get appropriate feedback from the program, e.g., “your microscope is out of focus.” They may try many permutations and combinations of options before resorting to solving the problem with the given math formulas.
  5. There are numerous correct solutions; when the student has arrived at one, the program tells them that they can proceed to the next phase or to print, if the project is now complete.

We believe that this is truly an educational process, where students learn the required curriculum content exactly when they need it, to solve a real-world problem. This process makes learning relevant, timely, open-ended (with multiple solutions) and engaging. Best of all, you have a real working microscope to show for your efforts!

If 3D printing is going to be successful in schools, the trick isn’t just in making user-friendly 3D printers; it’s about enabling project based learning tied to specific curriculum objectives. We believe simulation software, combined with easy-to-modify 3D models, is the key to success.

girls_stem

A Unique Opportunity

I have been in the technology field, specifically video games and educational technology, for more than 30 years, running three small businesses in LA and producing many games and apps. Throughout that time, I have witnessed firsthand the lack of diversity in our business, especially women, and its negative consequences for product design and development.

About a year ago I was introduced to Liz Ackerman Hicks, the dynamic principal of a new public school that is opening this August – GALA (Girls Academic Leadership Academy). This is LAUSD’s first all-girls school, for grades 6-12, and is focused on teaching STEM – Science, Technology, Engineering, and Math. It is a rigorous, college preparatory curriculum, and open to any girl in the district. The school is co-located on the historic campus of Los Angeles High School in mid-city. Additional classes are offered in art, architecture, and design, including an innovative Maker Lab with 3D printers. Project based learning make courses feel relevant by solving real life problems as part of the curriculum. Extracurricular clubs and activities include sports, music, robotics, and flight club (with flight simulators).

Cute girl showing homework on tablet.

Why is GALA an all-girls school? Research shows that girls do equally as well as boys in math and science in elementary school, but their grades and test scores falter in middle school and fall further in high school. All-girls schools, especially in urban areas, show higher graduation rates and higher enrollment and persistence in STEM courses.

My ask is simple. If you know of any girls who might be interested in attending GALA this August, there are still some openings for prospective 9th graders. Please help us get the word out about this wonderful and FREE public school alternative. Apply here.

It’s time to change the status quo, and what better way than through K-12 education? I couldn’t be more proud to be a volunteer for GALA.

grad_girl