How Augmented Reality can make education truly inclusive

When I met Cheryl Bayer, founder of Living Popups, at my office in Palo Alto, I was not sure what to expect. I felt a little overwhelmed by her insisting on flying down from Los Angeles just to meet me for a couple of hours. I wondered if it would be worth my time — and hers! It turned out to be one of the most fruitful and insightful meetings I had had in a long time.

Cheryl has an enviable bio: having worked and been instrumental in the success of blockbuster TV series such as Sex and the City, Friends, and Baywatch, she has proven to be a master storyteller. What inspired me was her mission to use technology, especially Augmented Reality (AR), to reach out to children with learning difficulties. Cheryl had left a lucrative job at Fox to pursue her passion in the area of Children’s Edutainment.

The topic of education was of deep personal interest to me — as a parent of a child on the autism spectrum, I was struggling to find ways to teach my 11-year-old non-verbal son Vivaan. Cheryl’s storybooks were interactive and when she showed me how she managed to embed AR in The Boxcar Children book, I immediately wanted to try it out with Vivaan. The Boxcar Children is a children’s book series originally created and written by American first-grade school teacher Gertrude Chandler Warner. With over 150 titles, the series is aimed at readers in grades 2–6. So, I ordered The Boxcar Children from Amazon.com and downloaded the *Boxcar App on my iphone.

I spent the next weekend reading the book to Vivaan. When your child is non-verbal, the biggest challenge is to know whether they’re able to follow the storyline and understand the details. The book had a picture at the end of each chapter. After reading the chapter I could point the app at the picture and questions would pop out of the phone; Vivaan could respond to them by pressing the correct answer on the phone. I tested this out on Vivaan by asking him specific questions and to my pleasant surprise he answered all of them correctly. This could be no fluke and it made me realize how much he understood cognitively. What had been just a black box suddenly opened up the world for me. Here was a technology that was truly inclusive to many learning styles. Technology gave me a way to understand my son!

So, what exactly is Augmented Reality? AR is an interactive experience of a real-world environment where the objects that reside in the real world are enhanced by computer-generated perceptual information. This is sometimes done across multiple sensory modalities, including visual, auditory, haptic (touch), somatosensory (pressure, pain or warmth) and olfactory (smell). AR can be defined as a system that fulfills three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects. It is important to note that AR is different from Virtual Reality (VR). In AR, layers of virtual objects are added to the ‘real’ surrounding environment, whereas in VR the surrounding environment is completely virtual.

AR has been explored for many applications, from gaming and entertainment to medicine, education and business. The tipping point was the game Pokémon Go (in year 2016) which pushed AR, a formerly uncommon technology, into mass adoption. The current uses of AR are in retail/hospitality training, military training and readiness, enhanced surgical training, property tours etc.

I asked Cheryl if we could use the technology to spread the message of Inclusion, especially amongst children. And that’s how we arrived at the idea of Inclu, the Augmented Reality mascot of the India Inclusion Foundation (IIF). The key purpose was to define the meaning of inclusion in a simple, yet interactive and playful manner. Many people have never met a person with disability. What if we could get Inclu, a person with disability, to enter your room and define inclusion in his terms!

We were able to build out the entire AR Inclu app in three weeks and started a social media campaign. In less than a month, the app has been downloaded thousands of times and people have shared their photos with Inclu on many social media platforms. I realized that the combination of AR with education and entertainment, addressing kids’ needs at their own level, engages them in ways no other trans-media product to date can.

The current pandemic has seen an increase in use cases for AR, especially for remote assistance and training, because of social distancing and lockdown requirements. As new technology brings out new use cases, I hope Inclu not only finds new friends but also very subtly changes the narrative around disability and inclusion.

(* Boxcar 1 and 2 is available in the App Store. Not only does it grab and keep a child’s attention, if a child gets a reading comprehension question incorrect, a character redirects and empowers the child to re-read the page)