So, the word “virtual” is used to describe many things. In 2020, we have virtual meetings, concerts, events, and more. I understand this, and it’s okay. The term Virtual Reality, which means a very specific thing, gets misused all the time, and I’m not okay with this.
I was at SXSW years ago, when 360 video was just starting to be a thing. During a presentation, panelists were asked if 360 video was virtual reality, and none of them could answer the question. The easy and correct answer is no. 360 video is it’s own thing, and it’s not VR. It surprised me that the panelists struggled with this question. Our early VR work at space150 (like the Polaris VR experience below), showed me what was powerful about Virtual Reality, and exactly what it was.
Polaris Victory VR (late 2014 / early 2015)
In the spring of 2016, just before the release of the HTC Vive and the Oculus Rift, I was at the SVVR (Silicon Valley Virtual Reality) conference in San Jose - an exciting time to attend this show. This conference covered Virtual Reality, Augmented Reality, and 360 video. This is when I realized this space was complex and nuanced, and needed education and explanation.
A good way to see the difference between the mediums of film and virtual reality.
At this point, I set about focusing on education and explanation of AR, VR, and 360 video. It was clear that our clients and partners needed to understand these platforms better to know how to leverage them. Virtual Reality is a full six-degrees of freedom sensory takeover. This is a fancy way of saying that you can look in any direction (three degrees of freedom) and you can also move in any direction (three more degrees of freedom). This is exactly how we humans experience the world, and it’s the things that is is completely unique to VR. Six degrees of freedom (“6DOF”) is what gives presence to VR. Presence is VR’s superpower, and what makes it feel like you are literally in a different world.
An example of a “fight or flight” response in VR (I’ve lost the source for this, if you know it, please let me know, I’d love to credit it, and learn more about it).
Many people are confused or unclear about exactly what VR is, and this is not surprising. Many products have been marketed as VR, when they are not VR. Examples of this are the Samsung GearVR, the Google Daydream, and the Oculus GO - all marketed as VR, but not providing the full 6DOF experience. All of these are headsets that allow you to look in any direction, but you can’t move through space - so you don’t experience presence like you do in VR. These aren’t bad products, just just aren’t VR. It also doesn’t help that all of these headsets generally look the same. People may think they have experienced VR, when they actually haven’t. The worst case scenario here is “yes, i’ve experienced VR, i tried Google Cardboard once”.
As VR has progressed, I’ve seen a few things along the way that have really surprised me. The release of the original Oculus Quest (March 2019) was one of these moments - a $400 standalone VR headset. No computer needed, no cables, just a simple, seamless experience. It’s not the most powerful system out there (desktop VR systems that are tethered to a powerful computer can render higher fidelity visuals), but what you give up in computational power, you more than make up for in easy of use and scalability. The original Quest felt like the first iPhone - it’s the standard for how things are going to go, and we’ll see rapid improvements. We’re already seeing improvements, in fact - the Oculus Quest 2 was released in October 2020, and it’s cheaper at just $300 - and significantly more powerful. We’re on a good trajectory. It’s easy to say that at $300, it’s hard to imagine a piece of consumer hardware at anytime in history that’s been a better deal.
I had another “wow moment” recently. With the release of the 2020 iPad Pro, Apple added a LIDAR sensor, which enables better augmented reality by using this sensor to scan and track the world around you. In the fall of 2020, Apple added this LIDAR scanner to the iPhone 12 Pro and iPhone 12 Pro Max models. In addition to using this sensor to understand the environment around you, it can be used to scan the world around you.
Scanning part of my house using an iPad Pro 2020 model with LIDAR. Different colors represent different surfaces (floors, walls, countertops, etc).
This got me to thinking - if we can scan environment using LIDAR, we should be able to view these places in VR, right? So I got to it, and spent about a day experimenting - and here is the first proof of concept.
At first, the video above looks normal. When you keep watching, you realize this is something different. This is a scan of my kitchen and most of the first level of my house. I scanned using the amazing Polycam app, optimized the mesh and textures using 3ds Max, and imported this into Unreal Engine and set this up for the Oculus Quest and Quest 2.
It’s early days with this, but it’s easy to see where this is going. This reminds me of the first photos I took with my original iPhone: this is amazing today, and it’s only going to get better. We’re at day zero, iteration one.
Amazing.
