Your Next Mac

October 24, 2021 · Milan Lajtoš

virtual reality, Apple, human-computer interaction, prediction

Nope, your next Mac won't be an iPad. Rather, it is going be a computer of a new form factor that is on the rise at the fringe of the gaming community. Yes, this post will be about Virtual Reality, which will change the whole computing world. Wanna know how?

What is a Reality?

Lets say that I put two giant displays in front of your eyes. Everywhere you look, you see an artificial image created from pixels so small that you can't see where one ends and the other begins. You can move your eyes left and right, up and down – your whole field of vision is being covered by this head-mounted display (HMD).

Now, let's put the best cameras on this HMD, and pass the image captured by cameras onto the display. You raise your hands in front of your camera-eyes and you look at them through the displays which are in front of your eyes-eyes. You can clearly tell these are your hands. You see the reality as it is.

Suddenly, a cute little dragon lands on your right hand.

It is small, soft and gentle. As the dragon crawls on your hand, you sense only a small tingling on your skin. You are twisting your arm and bringing it closer to your face, to get a better glimpse at this little creature. You clearly see it's scales deforming when it is turning here-and-there, the eyes have a deep green tint you haven't seen before, and the wings are a bit translucent, so you see glowing sources of the light behind them.

The dragon briefly looks at you, but it's attention is redirected towards some flying thing – by the sound of it, you know it is a mosquito. You and the dragon are both visually tracking the flight of the mosquito as it is flying near you. You sway your hand at it, and the mosquito flies away somewhere into the distance with an irritating buzzing noise.

You unknowingly scared away dragon's meal.

Without any apparent mood change, the dragon looks at you, spreads it's wings, yawns, and finds a cosy place on your hand where it falls asleep in a second.

Reality-Virtuality Spectrum

With a little sleeping dragon on your hand, you must think that it is a virtual, or synthetic creature. In other words, you know that it isn't real, even though the experience was believably real. How do you know it? For starter, you haven't seen a living dragon before – that is a big reason to believe it isn't real. You also know that I put an HMD on your head, so you were expecting some trickery. Is there any other reason you know the dragon isn't real? Probably no.

So what about the mosquito? Was it real or was it virtual? You have seen mosquitos in your life, and this particular mosquito was doing normal mosquito stuff. The obvious reason for saying it is virtual, is now out of the window. And the main trickery wasn't the dragon – of course, it was the mosquito. In virtuality, the mosquito was just as real as the dragon.

VR, AR, MR, xR

You have probably heard about various buzzwords that are describing what you have experienced with the mosquito and the dragon. Some people call it Augmented Reality, other call it Mixed Reality. Other names that are in use are Computer-mediated Reality, Simulated Reality, Transreality, or more broadly the Extended Reality. Everyone has a different name for these things, and if you want to be technical, there are differences, but for the sake of clear explanations, I am going to call it just Virtual Reality.

Better Way to See Pixels

The main enabler of VR is a new display technology. You have fancy high-resolution displays on nearly every electronic device you own and use – watch, phone, tablet, laptop, desktop computer, TV. All these existing displays are going away with VR HMD. Single question you have to ask yourself is this – why have 6 different 2D displays, when you can have a single one that is able to display 3D content? Clearly, in terms of the ability to display stuff, HMDs are the answer. Suddenly, any surface can become a display.

However, this doesn't mean that phones or tablets are overnight useless. No. They provide an ability to affect displayed content with a simple touch of a finger. We need these interactions, so keyboards, mice, trackpads, game controllers, touch surfaces, and every other input devices are more than welcome in the VR world.

So, with this knowledge, this is how a future workplace desktop PC will look like:

This type of a "desktop" computer is pretty compact, and it can display much more than a traditional two monitor setup. The limiting factor of screen real estate is now totally gone. Doing a single task that requires you to have several large windows open at the same time is now a breeze. Are you done with today's work and you want to play a racing game?

Or do you want to do a high-intensity interval training that does not feel like a mindless workout, rather like a kung-fu themed VR game? Or do you want to watch a blockbuster movie in an IMAX-like quality? Do you want a face-to-face talk with your friend that is currently on another continent?

Work, social, entertainment, or just everyday stuff, the answer is always this:

The (Higher) Middle Path

There are plenty of HMDs on the market right now, and they bring a different trade-offs to the table. The top of the line – ultra-high quality displays are usually bulky, heavy, expensive and requires you to have a super-powerful PC. This path is not going to win any large portion of consumers. On the other hand, there are less expensive HMDs that have sub-par display quality and the range of experiences you can have with it is rather limited. But there is also the middle.

The HMDs that are in the Goldilocks' zone of consumer VR technology have these characteristics:

standalone – you don't have to have PC to use it
under-powered – the computing power is not enough to play AAA games
6DOF camera-based tracking – the HMD has cameras around its body which detect how HMD is moving in the surrounding space with 6 degrees of freedom (position + rotation)
low resolution black-and-white passthrough cameras – to be able to see real world through the display
<100° horizontal FOV – human vision can span ~200° horizontally, while stereoscopic overlap is ~115°
<90° vertical FOV – human vision can span ~135° vertically
LED-backlight LCD – type of display that has limited contrast ratio, i.e. blacks are not that black
<20ppd – PPD stands for "pixels / 1°" – 20/20 vision is 60ppd, while theoretical limit is 20/8 vision which is 150ppd
90Hz refresh rate – how many times a second display can change its content, theoretical maximum of human eye is 1000Hz, however 120Hz is more than enough
Fresnel lenses – compact and lightweight lens that produce unwanted visual artifacts and distortions
no eye-tracking – the HMD doesn't know where are you looking
focal length at ~1.2m – stuff close to your yes will be blurry
manual IPD adjustment – "inter-pupillary distance" is distance between your eyes – you have to manually select the right fit
built-in speakers and microphone – you don't need any extra device to experience a full immersion
2 controllers included – how you actually control the experience
hand-tracking - you can use your hands to control the experience, but it is quite limited
usage as an external monitor – yes, but severely limited and no easy setup
~500g weight
2-hour battery
foam face padding – it collects sweat and is disgusting
sub-$1K price – anything above the $1000 is considered to be too expensive for mere mortals

This is the absolute minimum for a HMD to be technically decent. Of course, there are other questions, like how many games/experiences are available, if there is some hidden cost, software quality, privacy issues, availability, repairability, etc.

The Apple Way

So lets put on our thinking hats and imagine what an Apple-branded HMD would have compared to the absolute minimum-level consumer-ready HMD. Of course, these are just uneducated guesses mixed with wishful thinking, so beware:

There is no point of making a HMD without the guts of a computer, so your next Mac is going to be a standalone device. It is going to run on custom M-class chip with increased GPU core count, which is able to run AAA games.
There si going to be 6DOF tracking, but it is going to use combination of LiDAR and high-resolution color cameras. Tracking will be hardware-accelerated by dedicated coprocessor.
The horizontal FOV is going to be about 130°, which is not much compared to the 100° of the others, but is enough to be comfortable for most people.
The vertical FOV is going to be 90°, i.e. the same as the rest.
The device will have microLED display, which can deliver wide-spectrum with deeper blacks at 35ppd with 120Hz refresh rate.
The optics is going to be an aspheric custom-made lenses, while eye-tracking can be used to further enhance image quality for specific viewing angle. Also, it will support multiple focal lengths, so close objects will be sharp. If you wear prescription glasses you won't need them.
The HMD will automatically adjust to your IPD with readings from eye-facing cameras.
The speakers and microphone is going to be built-in, while you can easily pair AirPods for spatial audio.
No controllers included, but you can connect any 3rd party game controllers (PS, Xbox).
High-performance hand-tracking in combination with voice and gaze-tracking will enable you to control the UI as fluidly as possible. Built-in Universal Control feature which enables you to easily use all your existing Apple and HomeKit devices through the HMD.
It will weight 300g, and it will work for 5 hours on single charge. You can easily swap-out paddings (silicone, mesh fabric) that magnetically attach to the HMD and are actively cooled.
It is going to cost you $1,399 for the base model. You can choose from two sizes, three colors and 3 storage options – 128GB, 256GB and 1TB.
And of course it is going to be aesthetically pleasing. 😎

New Dimension to Experience and Create

Many future owners of this Mac have no clue about the potential of VR. They don't know that they will be able to experience Apple-branded movies, TV shows and sport events in 3D. (And by 3D I don't mean pseudo-3D TV/cinema garbage.) They also don't know that this HMD will enable game studios to remake favorite AAA games into VR, giving them extremely new feel. The future casual gamers also don't know that their favorite VR game does not yet exist. The future workers have absolutely no idea that they can work wherever they want to. Going to the cinema will feel like using a phone booth. Also, believe me when I tell you, that you will pay for AdBlock on your HMD. Talking to anime looking people, chickens, movie characters, or cute Memoji faces will feel pretty normal. Going somewhere by bike, foot, or car will be a whole lot adventurous.

And soon our virtuality will feel more real that our reality.

The year 2005 was awfully long time ago, but this still holds…