Author Archives: Ario

Augmented Reality and Schools – What’s Next?

What determines if an EdTech product is successful? The criteria are daunting. (1) It must teach something faster or better than is possible otherwise. (2) It must be relevant to the curriculum and teachers’ needs. (3) It must be affordable and accessible to all schools and students. (4) Ideally, it engages the student through experiential, discovery learning rather than rote memorization.

How hard is it to create this mythical piece of technology? Well nigh impossible, if looking at the wreckage of past innovations is any indication. I’ve written previously that the most successful attempts have been those hardware and software products that could be used by teachers in a flexible fashion, as tools to teach a variety of different subjects. Computers, tablets, Hypercard, Microsoft or Google office software, programming languages, graphics apps, etc. all make the list of successful technology innovations in classrooms. Their precise level of success, of course, depends on how well they are implemented in the classroom.

Now Augmented and Virtual Reality are headed for the classroom. How are they faring? It is difficult to justify the use of VR in the classroom, given the cost of most hardware. And while Google Cardboard is affordable, it is limited to presenting pre-rendered visual experiences in 3D. Is it that much more effective than watching a YouTube video? I doubt it.

And what about Augmented Reality apps for the classroom? I’m admittedly more of a fan of AR than VR. In its best incarnation, AR incorporates the real environment as part of the experience rather than create an alternative world as in VR. Some types of AR are possible with low end mobile phones, so cost isn’t necessarily an issue.  However, judged against the success criteria I listed initially, AR is sorely lacking.  It is not clear that most of the curriculum based AR experiences are more effective than other teaching methods.  And while there are some open-ended tools for creating AR experiences (e.g., Blippar, Zappar), they are focused on the consumer market, and not particularly appropriate for school use.

Meanwhile, AR is being rapidly implemented in industry, with use cases that would be equally helpful in K-12 schools and colleges.  Businesses are challenged with training millennials to take the place of retiring Baby Boomers, and they are finding that Augmented Reality is the ideal tool for any kind of procedural learning. Workers are  taught how to fix or assemble complicated equipment using step by step procedures that appear directly on machines. Similarly, workers are learning advanced manufacturing techniques, with operational guides turned into 3D displays.  Real time collaboration as well as asynchronous communication are possible using AR, with the remote expert providing help and guidance by leaving messages or drawings in the relevant location for workers.

How would that type of functionality fit into the classroom? Well clearly (and I’d like to be on the team that creates this), this mythical app would need to be designed specifically for schools, and include all the administrative, networking, and privacy functionality required. it would greatly facilitate distance learning.  A remote or in-classroom teacher could look at what a student is working on, e.g., 3D printing, robot assembly, chem lab, shop, frog dissection, circuit builder, etc. and write or speak comments, displaying instructions directly on the objects in question, in real time or asynchronously. Alternatively, two or more students could work collaboratively, in the same location or remotely, using this AR technology.

Some innovative developer is going to take the AR functionality being used successfully in industry today, and build a similar platform for high schools, technical schools, and colleges to engage the learner in new ways. I’d love to see a STEM-oriented tool for students to practice the engineering design process from problem to prototype while seamlessly collaborating and troubleshooting with the teachers and peers using AR.  It’s coming, and I, for one, can’t wait for the future to get here!

Augmented Reality in the Classroom – Lessons Learned

Many companies with glitzy AR apps have come and gone; many educational use cases have been tried and discarded. Is Augmented Reality just the latest in an endless stream of educational technologies that don’t add up to more learning?

Maybe. Beautifully rendered virtual images of the periodic table and human anatomy or spinning globes have failed to become standard features of the school curriculum. Volumes have been written about why, but it comes down to cost, convenience, and ROI. Educational AR apps tend to be difficult and costly to create and must compete with traditional “supplemental” school teaching aids. Students generally have to look through an expensive smartphone to see the virtual graphics and animations, making the interface awkward. While the initial experience engenders surprise and delight, there is little need, or desire, to repeat the experience. It’s also not clear how seeing a dinosaur “come to life” from the page of a textbook actually increases learning.

If you look at the history of educational technology, the technologies that ultimately earn a place in classrooms are used as tools first, rather than to directly deliver curriculum content. Consider Apple’s Hypercard, introduced in the late 1980s. Teachers loved it and similar products because they were easy to integrate into any subject matter. They turned kids into active creators rather than simply consumers of content. Plus they exposed children to more “real world” uses of technology, better preparing them for future jobs. Tools like Hypercard did more than anything else to ensure a permanent place in classrooms for computers.

The first interactive product I ever produced was Children’s Writing and Publishing Center, published by The Learning Company. It allowed kids and teachers to easily create newsletters, brochures, etc., similar to PrintShop but specifically geared to educators. It was a huge hit. My next product was another language arts tool, Mickey’s Crossword Puzzle Maker. Using Disney characters, kids could create and print out their own picture-word crossword puzzles. It was already clear, back in the 1990’s, that open-ended, tool based learning was the best way to insinuate technology into schools.

There are, of course, many tools designed to help educators create Augmented Reality content and experiences. The more popular AR authoring tools that don’t require programming are Zappar, Blippar, HP Reveal, and Metaverse. But even the simplest of these AR creation tools are made more difficult because they require the ability to make, access, and manipulate 3D graphics. (That’s the virtual part!) There is simply not a great option for K-12 schools, i.e., one that is easy enough to be used by teachers and students but sufficiently sophisticated to support mixed reality experiences that go beyond simple target based 3D pop-ups.

My sense is that the path that Google has chosen to win teacher acceptance for its AR/VR technologies will turn out to be the most productive. Google Expeditions is a great start, but students are still consumers of content, not creators. However, with Tour Creator, Google’s latest AR/VR product for schools, students grab scenes from Google Street View, and compile them into a Powerpoint-like presentation with text and voice. Neatly solving the dilemma of where to get and how to manipulate 3D assets, Google’s Poly and Street View provide 3D graphics and 360 video for most locations. Soon, Google says, students will be able to view their tours created with Tour Creator in their Expeditions app and on new Chromebooks that now include ARCore, Google’s AR platform.

But there is much more that can be done. Why not…

  • Add a story creator capability to Tour Creator, so students can add more and different kinds of assets and content to their tour?
  • Create open-ended AR tools that are designed for specific subject areas, like chemistry or physics? Augmented reality could be used to highlight the reaction when you mix together chemicals to form a solution.
  • Design AR tools for real-time remote communication? Businesses are rapidly implementing AR to train workers on the repair of complex equipment. Why not use AR similarly to help students building projects in Maker Labs?
  • Develop an AR game creator with a variety of templates that can be customized or a quiz generator that uses object recognition to teach foreign languages?

We’re at the Hypercard inflection point with Augmented Reality. Now we just need the right tools to convince teachers that this is an educational technology worth having.

A True Turing Test

When the first smart speaker was announced by Amazon in 2014, I was unambiguously enthusiastic about its potential with kids. I immediately approached Amazon, then later Google about creating skills/actions for Alexa and Google Home. Sadly, I was told by both companies that they weren’t interested in children’s content because of privacy/COPPA concerns. Since my company couldn’t count on support from the platforms, and there wasn’t any way to monetize our development efforts, I didn’t pursue the opportunity.

Fast forward to today. Amazon’s Alexa just released a new child friendly device, called Echo Dot Kids with a monthly subscription plan and parent friendly features. Google Home has a huge library of family-friendly content. What accounts for the turnaround? Late last year the Federal Trade Commission revised COPPA to essentially look the other way when companies collect voice recordings of children under the age of 13, citing that the “FTC would not take an enforcement action,” as long as companies use an audio file to transcribe a command and then immediately delete it.  Adding fuel to the fire, Amazon and Google have disclosed that families love the device, describing parents as “voice-assistance power users.”

Smart speakers, (also known as Voice Assistants/VA), have enormous potential with kids. They use Speech Recognition, Natural Language Processing, Artificial Intelligence, and Machine Learning to understand what is being said and how to respond. They engage and empower kids with endless audio interactivity. Smart speakers support a natural way of interacting that doesn’t require reading, starring at a screen, or interpreting interfaces. Unlike parents glued to their cell phones, a Voice Assistant always reacts to the child and says something, even if it isn’t actually an answer to the question they asked.  And there’s the rub.

Children neither speak nor think like adults.  What are some of the unique issues that arise when designing voice interactions for kids?

  • To begin with, we don’t really understand what children are thinking when talking to a disembodied voice. What is their “theory of mind” as they stand in front of a Google Home or Echo device?  This charming 2017 study from MIT Media Lab, “Hey Google is it OK if I eat you?”   revealed significant developmental differences. A younger child (4 years old) treated Alexa like a person, asking questions like “What’s your favorite color?”  and “How old are you?” Older kids (7, 8, 9 years) treated it more like “a robot in a box,” believing that the smart speaker could learn from their mistakes. Knowing that young children think of the smart speaker as a human gives one pause. Will they “take it personally” when the voice assistant continually responds incorrectly to a query?
  • We know that memory for audio-only information is relatively poor among kids, especially compared to reading or watching a story. This was one of the key findings of my doctoral dissertation research many moons ago and should obviously be considered in the design of audio only interactive content. Despite that, the first skill I helped design was a simple branching detective story, in the spirit of Encyclopedia Brown. We conveyed hints about “who dun it” throughout the story. Turns out there were way too many puzzle pieces to hold in short term memory. We realized that we had to write a simpler story, break it up in much smaller bits, and continually reinforce important information.
  • Egocentrism is a key feature of childhood. Kids have difficulty putting themselves in the place of another person, seeing the world from another perspective. It is not until children are about 11 years old that they begin to accept the limitations of their knowledge and understand that their knowledge is not the same as others. Asking a question of a Voice Assistant like Alexa, Siri, or Google requires them to adhere to some fairly strict conventions and to understand that they may have to adjust the way in which they ask a question in order to be understood. For example, when I asked, “How does an ice cream truck sound?” I got a charming response from my Google Home device (try it!). But when my four-year-old grandson asked what he thought was the same question, “Play me an ice cream truck,” we ended up with some rap music on Spotify.  Getting Izzie to rephrase his question, to think about how to ask it differently in order to be understood by Goggle, was a tall (impossible) order.  When we received a complicated answer from Wikipedia in answer to a question about cement mixers, Izzie walked away.

Creating a smart speaker/Voice Assistant for kids that can understand and respond satisfactorily strikes me as a true Turing Test, and one that we haven’t yet achieved.  While adding kid-centric skills/actions is a good first step, smart speakers are still more frustrating than they should be, both in understanding children’s speech and intent, as well as in their responses, or lack thereof.

The Five Year Pivot


“Ode” was created by Vincent Carrella, www.vincentcarrella.com

I was thinking about how to summarize this podcast about my life and career, and kept coming back to “Shit Happens, Pivot.”  Basically I founded three companies in the interactive business, then went through gut-wrenching change about every five years in order to stay solvent. There were no roadmaps, no mentors…so naturally I made every mistake possible.

Oftentimes pivoting was necessary because of changes to hardware and operating systems.  We not only needed to change our development tools, especially in the pre-Unity days. As we discovered with the transition from PC to mobile, along with new hardware comes changes to  distribution channels (app stores) and business models (free to play). This affected game design as well as game genres, and required a complete mind shift to game services. In other words, most of these pivots were excruciatingly difficult.

You can listen to the podcast, or if you are interested in the abbreviated version of the twists and turns, so far, in my 30-year career, here goes.

  • I pivoted from academia to business, because i couldn’t stand the politics of university life and I wanted to make a difference. Doing memory research with nonsense syllables didn’t seem like the way to “put a dent in the universe.”  (Steve Jobs)
  • I went from being a design consultant to creating my own products, and completely underestimated how difficult that would be. Let’s face it…anyone who comes up with a character called “Mutanoid” deserves to fail!
  • I developed CDROM products for kids, until that business imploded at retail. Turns out, selling games for $0 isn’t a sustainable business model. Reminds me of the poor health of the kids mobile app market today.
  • I pivoted to digital distribution and decided to target women customers. They weren’t playing a lot of games at the time, but I knew women (like me) loved detective TV shows and books. I figured that if I just created content that was more appealing to women, they would come. Serendipity struck, and I sat next to Dick Wolf (the creator of Law & Order) at a benefit dinner. I persuaded him to make a game, the first one licensed by NBC Universal.  We went on to develop and publish many more games based on TV shows, like Murder, She Wrote and Criminal Minds, and distributed them through downloadable portals like Big Fish Games.  It was an excellent business while it lasted, but after about five years, smart phones came along and decimated sales.
  • How hard could it be to make mobile free-to-play games? The game mechanics seemed simple enough (e.g., match-3), the graphics were mostly 2D, and the app stores handled the financial transactions. We got lucky early on, with a mobile game called Atlantis: Pearls of the Deep. Google loved it, and promoted it heavily. Quickly we had a couple of million players. And then what did I decide to do? Instead of continuing to develop content for our game, update it and service our existing customers, I made the classic mistake. I assumed we should immediately start work on the sequel. I clearly didn’t understand the “games as service” model, and threw away the best chance our company had of building a robust  mobile business.
  • While still designing and developing games for mobile, I next pivoted to Augmented Reality. I was intrigued with the technology, plus it was obvious that there would be lots of opportunities in education, in addition to gaming. We won a Google competition to create a product for their new AR platform, which launched us into the world of computer vision. We needed help, however, and the decision to pursue this challenging new technology necessitated further changes that ultimately led to my joining HitPoint Studios, where I am currently, happily, President.

One thing I must make clear before closing. While I made all of the mistakes chronicled here, I have been truly fortunate in the people I have worked with over the years. Some amazingly talented, and kind people, without whom I’d have no stories to share.

As Stanley Kunitz concludes in his wonderful poem, The Layers:

Though I lack the art

to decipher it,

no doubt the next chapter

in my book of transformations

is already written.

I am not done with my changes.

Advice for Fellow Toy Inventors

I just returned from my favorite conference of the year, Toy Fair. I still enjoy many traditional toys (skip to the end of the post if you want to see my personal favorites), but this year I attended, along with HitPoint’s CEO, Paul Hake, to get feedback on our first connected toy prototype. It was a hoot.

Paul and I walked the three looong convention floors for three days, lugging a big black box filled with a 3D printed play set, wires, lights, etc. All the coolest stuff is pitched behind closed doors; we finally made it into the inner sanctum! We got an earful, about the business in general and about our toy in particular.  Feedback ranged from, “This is the best implementation we’ve ever seen using AR,” to “Connected toys with AR have tanked at retail. We need to understand why yours will succeed when others haven’t.” Two other general rules that wannabe inventors need to be aware of include (1) The “plastic” sold to retail must function successfully as a toy, even if the child never downloads the app or has an AR experience; and (2) Every bit of technology that adds costs to the toy (like bluetooth, approximately $5), must be carefully justified (and otherwise ruthlessly discarded).

Are we discouraged? No way. Folks were very generous with their feedback, and I actually think we landed on an even better concept as a result of our Toy Fair experience. Stay tuned. Plus, I am convinced that 2018 is going to be a great year for AR gaming for kids (thank you Niantic and Harry Potter) as well as for AR connected toys (see below). Like every entrepreneur, I have high hopes that we will find a more receptive audience next year.

One AR enabled toy that I predict will rack up big sales this year is Merge 6DoF Blaster. It plays like laser tag. I tried out the demo at Toy Fair, and loved the fact that I could hide from virtual enemies by ducking behind virtual obstacles. The final peripheral will probably look similar to a Nerf gun, and hopefully less purple than the current version on the website. Of course it all depends on the games that come bundled with the device, but the concept is intuitive and fun.

Another favorite example of AR-driven connected gameplay was Hasbro’s Hero Vision Iron Man AR Experience. Undoubtedly inspired by Lenovo’s successful Star Wars: Jedi Challenges released last year, the Hasbro product is less versatile, but also much less expensive ($50 compared to $200). The mask comes with three AR targets that can be placed around in your environment. Each target triggers 3D buildings to protect and villains to destroy. It is a simple and well-understood gameplay pattern that utilizes the license in a rather brilliant way. It is also highly extensible, with the purchase of physical Infinity Stones for new powers. (Plus it will probably be applied to other superheroes at some point in the future. Wonder Woman anyone?) I expect Hasbro to do very well with this toy when it comes out later this Spring.

One final note about this year’s Toy Fair: physical toys will never go out of fashion, and some of my favorite companies continue to inspire and delight. (You’ll notice my distinct preference for German companies in the following list, but their quality standards are generally the best, in my obviously idiosyncratic opinion.)

I recommend: Ravensburger for puzzles and games; Playmobil for play sets; Crocodile Creek for puzzles; Mary Meyer for plushes; Melissa and Doug for anything, Lego for building, Crayola for creativity, and Haba for infant and toddler toys.

Enjoy and keep playing!

Hake in the Heartland

What a guy! Reprinted from the Heartland Series: Q&A with HitPoint Studios CEO, Paul Hake

The Entertainment Software Association (ESA) spoke with CEO Paul Hake about the advantages of being located in Greenfield, Massachusetts, and how his studio leverages the surrounding talent pool to focus on the technologies of the future.

Showcasing the geographic diversity of the video game industry, the Heartland Series features interviews with video game publishers, developers, and innovators from across America, highlighting the groundbreaking work and innovation they bring to every corner of the nation.

Q: Can you tell us a little about yourself? How did you get your start in the video game industry?

A: My father was a computer programmer for a financial company and he brought home a sweet IBM PS/2 and introduced me to programming when I was in junior high. We programmed games together, and throughout high school and college, my plan was to start a game studio. I went to the University of Massachusetts at Amherst (UMass Amherst) specifically to design my major around starting a game studio. I took computer science classes, art classes, and business classes. After college, I worked for Hasbro Toys in Pawtucket, Rhode Island in their visual studios department, editing toy demos and commercials and used that as a jumping-off point to work for Tiger Electronics which was based in Western Massachusetts. There, I started designing and developing all sorts of handheld games, plug-in TV games, and eventually PC games. So that’s how I got my start in the video game industry. That allowed me to build a portfolio, which led to more deals. Then in 2008, I basically took my studio, which was about eight people at the time, and merged it with another studio that was also in western Massachusetts to start HitPoint Studios.

Q: Can you give us a quick overview of HitPoint Studios? What types of games and technologies does the studio focus on?

A: When we started HitPoint in 2008, we were mostly working on PC download games. We did a lot of games with Big Fish Games, WildTangent Games, GameHouse and iWin Games, which were in the casual game space. A few years after that we started working with Microsoft on developing launch titles for Windows 8. We were one of two studios that developed launch titles for the Surface tablets and when Windows 8 came out, our games were bundled in with its operating systems. In addition, we worked with Disney to create games for Windows 8 and through those work-for-hire projects, we raised some money to develop our own platform for social and mobile games. With that platform, we launched our own titles, some of which we’re still operating, some of which we sold off.

We developed some mobile titles for clients who still use our platform and then about 18 months ago, we got started in the mobile augmented reality (AR) space. We began developing games initially for Google’s Project Tango devices, including our first AR game, Crayola Color Blaster. What we learned through the nine years HitPoint has been around is that the team really likes technically challenging projects and projects that are still a little bit nascent that show promise because most of our team are engineers and they really enjoy working on projects that are cutting-edge. When AR came around and Project Tango was an opportunity, it was a great fit for us and now we’ve made a name for ourselves in the AR space. We’ve launched a few titles and we have a couple more in the works. It’s about half the work we’re doing right now. The rest is some of our own games and other mobile game development projects on a work-for-hire basis.

Q: Could you tell us more about your recent projects? How do you think immersive experiences will impact the future of games?

A: We made a conscious decision a couple years ago to focus on AR because it seemed like it was a technology that would be more readily adopted by people who would be downloading our games and that was the bet we made. So far it seems like it’s paying off. Since then, Apple and Google, the two biggest players in the space, have released their own AR platforms for their devices, so we’re well positioned at the moment to not only be on the cutting edge in terms of the technology, but also there will now be hundreds of millions of devices worldwide that will have AR technology built into them a few months from now.

For our AR titles, we’re mainly focused on creating experiences that encourage exploring player’s physical spaces and moving around, instead of tabletop AR apps and games. One of the games we did was called Color BlastAR for iOS, where you have to run around your house, your room, or outside and it spawns AR characters in your environment, making it a get-up-and-run-around arcade game. That kind of immersion I think is going to be more powerful because you can see AR take over the space you’re moving around in, which is a whole new experience. The next phase for this game we’re working on is making it a multiplayer AR experience to allow for collaborative and competitive mobile AR play.

Q: You also focus on mobile games. In 2017, mobile games generated nearly 43% of the global market. Do you think the popularity of mobile games will continue to grow? Why?

A: About half of mobile phone users are playing games, and I expect we’ll see this percentage increase along with the number of mobile users worldwide. There’s a lot of room for growth, especially if you’re thinking globally. In addition, more people will become accustomed to spending money on mobile games. However, I believe there needs to be more ways for users to spend money on games outside premium purchases and in-app purchases. Right now, mobile game developers have premium, IAPs and ads as our ways to monetize games. I believe we’ll start seeing more subscription based models that will give developers another way to monetize in a form that consumers are already very accustomed to through HBO, Netflix, Hulu, etc. Hatch has been announced already as one streaming subscription platform and I believe we’re going to see more people using these platforms treating gaming like Netflix accounts. This will allow the mobile game developers to monetize their games a fourth way and continue to grow the overall mobile game revenue.

Q: Why did you choose Greenfield, Massachusetts, for your headquarters? Are there specific advantages this area provides to video game companies?

A: Western Massachusetts is a great location to be running a software company. There are at least five major colleges here, Greenfield is not far from two of them and not far from Boston. There are a couple colleges in Vermont with computer science and art programs. Worchester Polytechnic and Rochester University are also not far. So, we’re in a really nice location for all these schools that are putting out really talented engineers and artists, so it’s a great spot to be for that. We weren’t always in Greenfield as we bopped around the valley a little bit, but we just started a video game co-working space in Greenfield and we found a spot we really like. The city has its own municipal internet connection that’s high-speed and it’s near the rail service and there’s a lot of great places to eat, hang out, and relax. We’ve worked with other studios that are in Boston or in other major cities that all have issues with parking, commuting, and extremely high costs of rent, which we don’t have to deal with out here, but we also have a lot of talent coming out of the colleges. Once people come out here, they really want to stay and it’s our open secret that Western Mass is a great place to recruit because the quality of life here is fantastic.

Q: Do you usually hire from local universities or are your employees from around the country? Are there specific areas of study you usually target when hiring new employees?

A: More than half of the people at HitPoint went to one of the five colleges here and the other people we’ve brought on are from other studios in Boston, who either commute out here or work from home. We have a very flexible work-from-home policy. We have a couple other people in our office in Los Angeles, another employee works from Buffalo, New York, because he has family up there, but our main hub here is in Greenfield. What’s really nice is UMass Amherst has a career fair, which we’ve participated in. There are two people who work at HitPoint who have been adjunct professors at Hampshire College, so we have a great connection with that college. They have a great game-focused program there, and in addition, we work with the local colleges to host a Global Game Jam and we actually have one here next month at our co-working space. They’ve been great opportunities to get juniors and seniors in college to get to know us. We get to see them, get to see their work, and basically keep them in mind and keep in touch with them after they graduate. We’ve done stuff like that and we have several employees, myself included, who have spoken at the local colleges dozens of times, giving talks about different career paths.

Q: What do you like best about your job?

A: I studied computer science and art, and I’m not that good at either of them [laughing]. I’m an ok programmer and I’m an ok artist, but what I do like doing is organizing the teams or really talented people we have here and watching them be successful in their product launches. What is really nice about how HitPoint is structured and operated now is I have to spend less time doing hands-on management and overseeing things because the studio has matured so much over the past eight years. We just launched a Facebook Instant Game for the new Star Wars movie, which got a lot of great coverage, but I happily had very little to do with that. It was great to just tee up the project for the team and see where they would run with it. Things like that are what I like most about the job; making sure the right people are in the right room or Slack channel, on the right project, and giving advice where I can.

Q: What is your favorite video game of all time?

A: That’s a tough one but it’s probably Myst. I had a very limited amount of game time available to me growing up, and we had a strict no console policy at our house. But Myst was one that I played multiple times, and really shaped the kinds of games I like to design and play today. It had great puzzles, a strong narrative that wasn’t in-your-face, and for the time it had amazing graphics and groundbreaking animation, music, and audio. That game probably got me into video games more than anything.

Q: What advice would you give to aspiring video game developers? Which skills should they invest in today to break into the video game industry and become successful?

A: This is one topic that I am asked to speak about at colleges a lot. For me, it’s always important for artists and engineers to be focused on the fundamentals. For engineers, make sure you have a strong math background, traditional computer science skills, not just focusing on the latest, hottest game engine and make sure you understand the fundamentals of engineering because those things don’t change. The game engines change, the languages change, “what’s hot” changes, right? I’ve seen a lot of colleges get tripped up by that by having their engineers focus their curriculum on the Unity game engine because that’s what’s hot right now or on the Unreal game engine or some other game engine and the students will have a decent portfolio on that game engine, but they don’t have a good base on which to build.

Same thing with art, too. A strong traditional studio art background, understanding lighting, colors, perspectives, and concepting. We’ve hired some artists from the Savannah College of Art and Design (SCAD) where they have a strong traditional art program, and some of the best artists we’ve worked with have come out of that school.

The other big thing is having a portfolio that shows off only your best work, even if it’s just five pieces on your website. If there’s something you’re not proud of, don’t show it. It’s definitely quality over quantity. For engineering applicants, I don’t think you can go into a job interview or expect to get hired when you have a game you worked on, but can’t send over the sample code and a playable build. We receive a lot of resumes that show what games people have worked on but are lacking the sources because it’s on the computer at the school they went to or it’s on a repository they no longer have access to or some other reason. That’s not going to work. You have to have games you worked on, even if it’s in the Apple Store or some Android APK that we have to side-load, students need to have sample apps available with details of what their role was in the development of that app or game. That’s the number one thing you need and you’d be surprised at the number of people who don’t have that when they apply.

A Very Merry Star Wars Xmas

Many of you may already know that I joined HitPoint Studios as President in October. It was a big change for me, given that I was the founder and CEO of Legacy Games for 19 years before that. So I thought that it might be time for an end of the year “how’s it going so far” report to readers of my blog.

In a word, great. It’s been a hang on to your hats thrill ride, with most of the company’s focus on either new platforms or new technologies.

– The really BIG news first. Today Disney released Star Wars: Porg Invasion, a fun social game that HitPoint created. CNN calls it “adorable.” It’s our first Instant Facebook Game, but won’t be our last. We’re pushing the limits of what you can do with HTML5 and Messenger!

– In other gaming news, we’ve launched the first four episodes of Adera on Google Play, a beautiful mobile game. It will soon be out on Hatch, an exciting new subscription service from Rovio, as well as multiple other platforms.

– Our Augmented Reality work-for-hire business has tripled in the past 15 months, to now include games, on ARKit and ARCore, as well as some exciting business applications. I describe, obliquely, some of our more confidential work with retailers in this recent blog post.

– And last but not least, we are working on four different connected device/smart toy projects, all of which include Augmented Reality features. Three are for consumers and one is B2B, where there is a strong appetite for customized, highly technical solutions…right up our alley.

That’s my “Xmas Letter” news. I wish everyone a peaceful, healthy and happy holiday season, and a prosperous 2018.  (A little less busy and tumultuous new year would be nice too!) May the light be with you!

The Future of Retail is Here

“The future is here, it’s just not evenly distributed yet.”
– William Gibson

The future is here and retailers are taking note.  We are in the midst of a major transformation in terms of how we shop, thanks to our ability to layer digital content on top of the real world. Combined with our increasing ability to turn past purchasing behavior into smart product recommendations, customers will soon be able to visualize in-store personalized promotions, coupons, and variable pricing alongside their favorite items on the shelf. The dual technologies – Augmented Reality (AR) and Artificial Intelligence (AI) – when paired together, will reinvent retail. 

When Augmented Reality is used in retail, the virtual and physical worlds are combined. Because the technology now exists to easily digitally map the interior of a store and know where every product is located, the customer can be directed step by step to the exact product they want.  Furthermore, when this location-based intelligence is combined with knowledge of the customer and their past purchasing behavior, we can ensure that their in-store experience is relevant to their preferences and more likely to lead to purchases.  

While there are many possible future scenarios regarding AR, AI and retail, my company, HitPoint Studios, has been researching which use cases resonate with customers and are technically feasible given current mobile devices. Retailers, however, are not all equally prepared to take advantage of what we learn about customer behaviors. From the get-go, companies like Amazon have a built-in advantage, compared to “legacy” retailers. Imagine how Amazon can and will transform the retail shopping experience, given their extensive knowledge of the specific customer and the ability of AR to display personalized information on top of the physical world.  

Amazon is already exploring what experiences and information the customer wants in their retail locations. Just in time for Thanksgiving, they announced that Amazon Prime members will be able to buy turkeys at a reduced price at Whole Foods stores ($1.99/lb. compared to 2.49/lb. for other customers). In the not too distant future, we will download an Amazon AR app and be able to see relevant special pricing and promotions, simply by pointing our phone camera over items in the freezer case or on shelves. Similar to how Amazon adjusts pricing on the fly online, the store turkey will cost more the closer it gets to Thanksgiving, and drop dramatically after turkey day. Furthermore, because of our online purchase patterns and the use of AI (Artificial Intelligence), we will be offered personalized discounts in the store. For example, I order avocado oil by the boatload on Amazon.com.  Amazon knows this and will offer me a virtual coupon as I walk past the oil shelf in Whole Foods.  Or, if avocado oil is out of stock on the shelf, I’ll be prompted to order it online and have it shipped to my home.  Similarly, customers will expect to access online rankings via Augmented Reality of every physical book sold in Amazon retail bookstores. And the transition between online and retail ordering, for digital or physical books, will be seamless.

Newer retailers, many of whom started out as online businesses, may have an easier time merging the online and physical world within their retail locations. Stores like Warby Parker and Sugarfina are not only “cooler,” according to millennials, but they know more about their customers. In addition, because their store locations are newer and often utilize the same inventory and planograms, mapping their physical interior and identifying the location of every item is a snap compared to older stores. Finally, I expect these formerly online-only stores to use in-store AR games and ties to social networks to attract younger buyers who want to share information and have a fun experience while shopping.

How do legacy retailers compete?  They are at a decided disadvantage, certainly. First, they need to know their customer.  Do they gather individual customer information, like purchase history and wish lists, that they can link to the customer who walks into their physical stores? How accurately do they track inventory and are products located in the same place in every store? Is there consistent pricing across stores? And more basic yet, do they have wifi throughout the store?  All of this is necessary information and infrastructure before AR can work its magic.

Someday your local retail store will know as much about you as Facebook, Google, and Amazon. Your online persona – your friends, preferences, purchases, and history – will be reflected in what you see and experience at your neighborhood mall. This is the promise and power of Augmented Reality and Artificial Intelligence, but its full implementation will probably arrive at some retailers sooner than others.

If you’d like to know more about the exciting AR projects we are working on at HitPoint, just drop me a line at alehrer@hitpointstudios.com.

Play Patterns, Character Interactions, and AR

Do you remember the Burger King web game, Subservient Chicken? A man in a chicken costume performed a wide range of actions based on the user’s input, showing pre-recorded footage. There were more than 300 commands that the “chicken” could follow. I remember watching a 7-year-old type, “Show me your butt,” and of course the subservient chicken did just that. It was hilarious.

This type of gameplay appeals enormously to kids because, at its core, controlling another character’s actions is highly empowering. Providing them with experiences in which they are the “boss” of another person, whether digital or real life, is immediately appealing. (I know many grownups for whom it works similarly.)

I believe that controlling increasingly life-like virtual characters will be at the heart of the most successful Augmented Reality (AR) experiences, and that classic play patterns will point the way towards satisfying gameplay and interactions.

With the advent of ARKit (Apple) and ARCore (Google), we have a new generation of 3D virtual characters to enjoy. Pet simulators are popular…surprise, surprise. Dress, feed, and play with Tamagotchi–like pets. The big difference is that these characters appear superimposed in your physical environment, as seen through your mobile phone. Keep up their “happy” meter, or they’ll run/fly away. These new AR virtual pets seem more life-like because they appear in the real world, and thus even more compelling to children.

You also control, albeit indirectly, virtual characters in HitPoint/Legacy’s new first-person game, Color BlastAR, on Apple’s ARKit. (A BIG update just launched, with extra-scary Halloween-themed graphics.) This fun walk-around AR game for kids of all ages combines tag and paintball with dragons, gnomes, yetis, zombies, and more for a very active experience!

Working on Color BlastAR has encouraged me to draw on other design inspirations, like classic outdoor multiplayer games, when thinking about virtual character interactions. What are some other interfaces, environments, and gameplay patterns that are relevant in designing an AR game?

  1. Back to Subservient Chicken. What if we could directly control virtual characters, with simple commands (voice or text) like “turn around” or “touch your toes”? Could we re-create the game red light-green light with virtual characters? Could we control them with music, e.g., the character dances until we turn off the music, or as in musical chairs? Or perhaps we control them with our movement. As long as we continue to move our arms, the character will move.
  2. Similarly, what if virtual characters could interact with each other, not just with the player? In Color BlastAR, I’d love to add a new gameplay mode – freeze tag. Once you color in a creature it remains frozen until another creature touches it, and unfreezes it! Meanwhile, you are still racing around trying to color all the zombies, orcs, dragons, ghosts, etc. before you get “chomped.” Would be frenetic but super fun!
  3. I wish virtual characters were able to interact more with their physical environment. Unfortunately, occlusion doesn’t work well with ARKit and ARCore…yet. Characters walk through each other as well as through physical objects in their path. But even before occlusion is solved, the phone still knows something about the environment. We could use day/night, inside/outside, GPS data, etc. to make characters smarter about where they are and change up the gameplay accordingly. Maybe we make the ghosts in Color BlastAR only appear when you play outside at night?
  4. Is it possible to combine target based and world-sensing AR so that virtual characters can interact with signs and symbols on the ground? I’d start with a virtual robot game, where it reacts to the symbols I draw in chalk (e.g., arrows) on the ground. If the robot steps off the correct path due to faulty programming logic, they disappear! (Naturally, you could control a virtual robot through programming from your phone as well.) Next we’ll be playing hopscotch with a zombie and designing mazes for our virtual friends!
  5. What if virtual characters were invisible, while playing Hide and Seek or Marco Polo? You follow puzzle clues, both visual and auditory clues, until you get close enough to reveal the hidden character (who then pops out and scares the s— out of you!) Or perhaps you find the invisible character by throwing color or light around, and if it intersects with the character, that part of them is now visible. I’m smiling just thinking about it.

I haven’t discussed sports games with virtual characters, which could be massive, especially once AR Cloud and multiplayer is implemented in ARKit and ARCore. I’d love to let kids play Color BlastAR together, teaming up to paint the creature horde in tandem. (It’s on the road map.)

How important do you think interacting with virtual characters is to future AR gameplay?

There is Nothing Permanent Except Change*

I made a BIG career decision, to join HitPoint Studios. I have worked with this talented team in Massachusetts on four projects already, and am convinced that their tech and software development skills are incredibly well suited to pursue massive opportunities in Augmented Reality. Wish us luck!

I also want to say that I am incredibly grateful, at this stage of my career, to be able to do what I love best – envision and help build the very best games and experiences that we can create. We truly are at the beginning stages of what is possible, and I hope to contribute to reimagining learning and fun in a virtual world.

*A quote from Heraclitus