How does the nature of play change in a world with augmented reality, when your device knows where you are and everything around you?

Pokemon Go broke through gaming’s summer malaise in a spectacularly dramatic fashion, due to its revered brand, fun collector-style gameplay, and “gee whiz” location-based augmented reality integration. But one of the most interesting aspects of Pokemon Go is, in my opinion, how it provides a glimpse into unique play patterns that are enabled by new technologies. Where do we go from here?

The Play Observation Scale (POS) is a good general purpose framework for thinking about the types of play and how it changes over the course of child development: functional (characteristic of very young children), construction, exploration, dramatic, rough and tumble, and games with rules. Within this, play can be solitary, parallel, or group (with one common goal).

Let’s focus on construction play first. (I will address other play patterns in subsequent blog posts.) Traditionally, this refers to assembling blocks, sewing doll clothes, building a fort, and more. Any time a child is a “maker” with an objective, that is considered constructive play. What happens when a child builds something with virtual items? Given AR possibilities due to 3D powered technologies like Google Tango, we now have examples of virtual items actually interacting with physical objects to create an entirely new experience. For example, in Woorld, you can add a spigot to a table top, turn it on, and watch your room fill all the way up to the ceiling with virtual water. To empty the room, you simply add a virtual drain to the floor and watch the water level go down. So much fun!

Here’s an idea. What if there were an AR app where kids power their own virtual machine capable of creating magical virtual objects of any shape and size? Once the machine has been constructed to the player’s satisfaction, a simple touch of a button or lever shoots out a stream of… what? You won’t ever quite know until you try. What will come out the end of the machine will be related to the components that have been added and used previously. For example, let’s say you added a Chicken Button, a Time Machine Rewinder, a Color Randomizer and an Anti-Gravity Extruder, which results in…flying purple eggs! You can move, drop, throw, and scale all the virtual objects created, filling up your room to the ceiling.

This game idea sounds like a dream to someone like me, who for 30+ years has championed the design of products that support open-ended constructivist theories of learning.  Of course it would be even better if the child could create some of the components and not have to rely on just mixing what is in their digital catalog. That would be “Crazy Contraptions” on steroids!

What are the benefits of adding AR to construction play, and is there a significant difference between constructing with virtual objects compared to real ones? Digital items can be easily edited and changed, unlike physical objects. Size them up or down and place them anywhere…even on the ceiling! They are vastly more adaptable than physical objects and allow for creative combinations not possible with real objects.

The disadvantage of virtual objects, and it’s a big one, is that there is currently no haptic feedback with mobile AR devices like Google Tango and precious little with VR either. You don’t feel as if you are actually touching something; you don’t have the experience of twisting and manipulating the object. You can’t feel the screw loosen with the screwdriver, or feel the hammer’s impact on your arm. For children, who learn initially through direct sensorimotor experience with the world, this is a significant issue. The younger the child, the bigger the problem, although the lack of haptic feedback in AR/VR, especially when the experience is mechanical in nature, impedes learning at every age.

At this stage of the technology’s evolution, assembling your new Star Wars Lego X-Wing Fighter with real blocks, compared to virtual ones, would be easier and more satisfying. But the finished product couldn’t launch into hyperspace, lost in clouds of stars and planets, like it could in AR. Perhaps the answer is a combination of physical toy with AR? To be continued…