Augmented reality could be the geology classroom’s killer app

One of the most important challenges in teaching geology is bringing the outside world into the classroom. During a pandemic, obviously, an inability to safely bring students into the classroom doesn’t make that any easier. Fortunately, digital tools can provide new ways to access the world beyond whichever room you find yourself in.

Geology is a very spatial science and can require a lot of 3-D visualization. Simple physical models (not to mention rocks) have long been used to aid teaching about things like faults or crystalline mineral structure. But these things can be surprisingly costly and occupy a surprising amount of storage space. This is an obvious place where technology can come in, serving up an endless variety of objects, simulations, and real-world data—if there’s an easy way for students to access it.

Augmented reality (AR) visualizations are increasingly capable of delivering on that promise. Ars talked to Martin Pratt about his work as part of a Washington University in St. Louis group that is developing apps for classes, both for specialized devices like Microsoft’s HoloLens and for the phones most students already have.

“You want to represent that data, not in a projective way like you would do on a screen on a textbook, but actually in a three-dimensional way,” Pratt said. “So you can actually look around it [and] manipulate it exactly how you would do in real life. The thing with augmented reality that we found most attractive [compared to virtual reality] is that it provides a much more intuitive teacher-student setting. You’re not hidden behind avatars. You can use body-language cues [like] eye contact to direct people to where you want to go.”

While completing a Ph.D. in seismology a few years ago, Pratt had taken an interest in data visualization along the way. He took on the challenge to work on creating AR apps, starting with one (iOS only) that allows users to explore up-to-date USGS earthquake data in three dimensions rather than just a 2-D map. You can much more easily understand the geometry of a tectonic plate boundary this way, or even explore the pattern of foreshocks and aftershocks around an earthquake.

Working with the Unity game engine, Pratt (and co-creator David Lie-Tjauw) also put together a flexible app called GeoXplorer (for iOS and Android) for displaying other models. There is already a large collection of crystalline structure models for different minerals, allowing you to see how all the atoms are arranged. There are also a number of different types of rocks, so you can see what those minerals look like in the macro world. Stepping up again in scale, there are entire rock outcrops, allowing for a genuine geology field-trip experience in your living room. Even bigger, there are terrain maps for landscapes on Earth, as well as on the Moon and Mars.

And it’s not just geology. Due to interest from people in other fields, Pratt has found existing datasets to plumb, like models of proteins, art, and archeology.

For now, this is a solo experience, but Pratt is working on building shared spaces into the app. An instructor could scan around the room and create a reference frame for the space. Other users could join the session from their devices, downloading that reference frame and matching it to their perspective. With this done, the students would all be looking at the same virtual object placed and controlled by the instructor, just as if they were holding up a model in the front of the room—shifting to a collaborative experience.

The group is simultaneously building a community database to hold a growing catalog of objects accessible with the app. For data that’s already digital, that’s a simple operation. Turning pieces of the physical world into digital models is a different operation. Thankfully, this has gotten much easier recently. It’s not necessary to laser-scan objects with expensive instruments. Using “structure from motion” photogrammetry software, you can simply process a number of images from different angles to make a model.

“We have a little light-box setup in the lab, which has diffuse light to show off the hand sample that you want to measure,” Pratt said. “It’s on a little turntable, so you just take a photo every 10 degrees and do 36 different photos all the way around at one level, then move the model a little bit and do another 36… And try and collect the entirety of that hand sample.”

The size and resolution of these objects are currently constrained by the ability of our devices to handle rendering them at the desired refresh rate. One thing that’s coming for these systems is adaptive resolution capabilities that load more detail as you zoom into a section of an object. This has a variety of applications; NASA’s Mars rover teams, for example, could use this to walk around in imagery from the Red Planet while retaining the ability to look closely at things. But it will also open up even more options for the classroom.

Going beyond static models, some interactive simulations are in the works as well. Pratt is working with glaciologists at Columbia University on a glacier simulator that allows you to grow and shrink a glacier on virtual terrain—or even on your desk or stairs.

These are all practical and unique tools for teaching and learning, particularly as they don’t require expensive new devices that have to be shared. (Although having a HoloLens certainly has its advantages in the UI and controls department.) And as more people find value in this technology and contribute new models of objects and places, its usefulness can grow.

“[AR] will hopefully sort of trickle down in a few years’ time to grad students and undergrads saying, ‘Oh, I don’t have to just use a laptop anymore. I can actually show this three-dimensional data that I’m trying to study and understand in a different way with a device that I have sat right here anyway,’” Pratt said.

You don’t need to be a student at all to have some fun with it, of course. Anyone can feel free to spin up a lunar stroll or summon exotic crystals to your table. You know, like a wizard.