“VR and AR will eventually converge, and smart glasses will take over our digital interactions.”
Carlos López (Founder @ Oarsis)

Augmented Reality (AR) is a process that uses technology to overlay digital content on real-world objects.  The digital content can be provided by, smartphones, glasses, and screens.  While AR is still an emerging technology, the buy-in from major companies like Microsoft with the HoloLens, WebAR support in Google Chrome using ARCore, and Apple’s augmented reality development kit ARKit, likely mean this technology is here to stay.

While the form factor used in AR will undoubtedly go through multiple iterations the primary function overlying digital content will remain constant.  AR is a great place for higher education to embrace technology and stay current rather than playing catch-up.  While wearable AR tech is not yet coming place, we can use the near ubiquitous smartphone with augmented reality.

There are already educational AR tools developed both inside and outside of education.  The Dinosaur 4D+ flash cards by octagon studio bring Dinosaurs to life.  Using an app installed on an Android or Apple device the flash cards allow you to explore and interact with the cards, as you can see here.

The International Society for Technology in Education (ISTE) has a blog post by Larysa Nadolny Worksheets for the digital age: AR interactive print.  The author gives a brief overview of the creation of these AR worksheets using existing technology.  Case Western Reserve is using AR to help teach anatomy, using the Microsoft HoloLens.  Students can see the anatomical process in active 3D.

Publishing companies are also starting to use AR in their books.  Carlton Books has two categories of AR books an educational category including titles like Explore 360: The Tomb of Tutankhamun and iExplore – Bugs that use AR apps to bring the content to life and let the readers interact with it.  They also have a new category of fiction novels they are working on; the first is The Ghostkeeper’s Journal and Field Guide a book that uses AR to enhance the story and engage the reader.

Many companies are producing AR books. Currently, the publishers are mostly focusing on the children and youth market.  These books have evolved from some simple animations like moving gears and simple 3D animals to full multimedia that include animations, sound, and interactivity.  Some of these books like the previously mentioned The Ghostkeeper’s Journal and Field Guide were written to include the book and its AR content as part of the story.

I have previously discussed how storytelling is a powerful educational tool (you can read about it here), I wish it was used more in textbooks.  If AR can enhance storytelling like these publishers are suggesting it should also enhance learning. While some people think the AR in books is gimmicky, I think anything that increases engagement with books is good.  Also, with regards to AR being gimmicky while Arthur C Clark said: “any sufficiently advanced technology is indistinguishable from magic” technology doesn’t have to be “magical” to be effective in learning.

This discussion of AR and books brings me back to the idea of textbooks.  The addition of augmented reality to textbooks can enhance education.  Let’s start by thinking about the basic content in a textbook.  We could add something simple like sound.  Imagine a music appreciation course; the textbook could describe techniques used in improvisational jazz.  Say for instance arpeggio, where the musician plays the notes of a chord one after the other instead of together.  Think how much easier this would be to understand if the textbook could play clips of music with and without arpeggio.

In biology, we often talk about how seasonal changes affect the local ecology and behaviors of organisms.  A great example of this is the Amazon Floodplain forests.  A large area of the Amazonian forest that is flooded every year in the rainy session when the Amazon river is overflowing its banks. Textbooks will often show flooded, and dry pictures to show the effects of the flooding.  With AR you could show a time-lapse video of the flooding and retreating water to get a better idea of how the water affects the landscape.

Something I remember from my days as an undergraduate in chemistry and biology is the difficulty students have learning to translate a 2D model into 3D.  Molecules are 3D objects when writing about them; we need to represent them on paper.  A simple model would be the wedge and dash model used for methane below.

In the diagram, the solid wedge means the atom is projecting out of the paper towards you while the dotted wedge means the atom is projecting away. I was one of the lucky students I have always been able to picture the 3D shape of from these drawings rather easily. However, I have known a lot of people that have real trouble seeing the 3D form.

Now imagine if the textbook had AR we could design interactions that not only projected the molecule in 3D but let the students manipulate, rotate, and zoom in and out to examine them.  AR projections would be especially useful when you get into more complex structural issues like stereochemistry, were molecules have the same formula but differ in their shape.

A textbook on public speaking could include actual audio and video clips of famous speeches.  A math book could include video clips were professors solve example problems with explanations.  We already know that publishers are taking advantage of AR especially in the case of books for young audiences.  However, AR textbooks are starting to appear, Introductions to Graphics Communication is a college-level textbook using Ricoh’s Clickable Paper.  Publishing companies in Japan have released textbooks with AR; you can read about them here.

Even with the availability of many AR platforms some of which are Augment, Blippar, HP Reveal, Daqri and Layar that offer educational pricing.  I have not seen any Open Educational Resource (OER) textbooks with AR content even the textbooks developed with large federal or privet grants.  In addition to whether governmental and privet organizations will be willing to pay to update these OER textbooks in a few years, are we also going to end up in a situation where we have different classes of textbooks? Is there going to a case where if you can afford it you get a different textbook?

Augmented Reality is a technology that higher education needs to embrace.  We need to develop not only resources using AR but the tools, preferably in a free and opensource platform, we can use to incorporate into any resource where it makes sense.  Textbooks are a resource where AR makes a lot of sense.  Like I have said before we are in the middle of a revolution regarding textbooks it is critical that we don’t focus on just one aspect of the textbook.  We need to think about what we want a textbook to be in total, and one of the things we should add is AR.

Thanks for Listing to My Musings

The Teaching Cyborg