Qualcomm said what humanity will experience when the Snapdragon 845 breaks into virtual space. Or how not to hit the wall and leave the window while wearing VR glasses.
Hugo Swart, senior director of product management at Qualcomm, is generous with compliments to himself (hereinafter, his comments are in italics).
Hugo is happy with everything
Imagine what the world will look like when the boundaries of the physical and digital world are blurred thanks to unlimited virtual reality (VR) and the improvement of existing objects around us (AR).
At the Augmented World Expo earlier this year, we came to the conclusion that 'XR' is a new term that is already being used to describe the sum of AR, VR and everything in between. XR is a fast-growing mobile market, according to TechCrunch, with a combined market of $ 108 billion by 2021 if VR and AR are combined.
In recognition of this, we have released over 20 XR platforms for Head Mounted Display (HMD) and XR-compatible smartphones to our customers, and the development of new concepts and devices is happening right now. We work closely with some of the world's most influential XR-related companies, including Google HTC Vive and Oculus. We also support Facebook, Samsung, ODG and many more. In addition, we have a Virtual Reality Acceleration and Manufacturing Acceleration program that supports customers to rapidly commercialize with reference designs, qualified ODMs and technology co-authors.
The above should be categorized as 'for investors and customers', but earlier this year Qualcomm is just splashing information and answering more interesting questions.
How not to stick your head into a real wall while in virtual space?
When we are in virtual reality, our device must be able to create and update a map of the surrounding space in real time and at the same time track its location inside it.
Stage 1 – Image Capture
To solve this problem, the Snapdragon 845 is equipped with a 6-position scanning and navigation system (6DoF with SLAM). It is impossible to name this complex of devices otherwise, and the English term Room-Scale is not very suitable in reading it in Russian.
Stage 2 – Scanning the space and building its 3D map
With the Snapdragon 845 '6DoF with SLAM' system, you will see obstacles and avoid them when you get too close. You can also calculate the size of a room, scan objects in it to avoid them, and integrate real-world objects into your virtual world. If you are 'Trekkie' (the slang for a fan of science fiction series), then you will notice some resemblance to the holographic room on the Enterprise (holodeck) spaceship:
Stage 3 – Overlaying the background on the physical boundaries of the virtual space
Qualcomm Snapdragon 845 is capable of transmitting an image at 4 million pixels per eye. To understand the issue – in front of each eye there will be a monitor with a resolution of 2560 × 1440 pixels (3.7 MP). To implement this difficult computing task, one central processor is not enough, all computing units, including the signaling Qualcomm Spectra 280 ISP and Qualcomm Hexagon 685 DSP, contribute to the Snapdragon 845.
Stage 4 – Applying textures to furnishings. You can sit on a stone and admire the mountain views
Putting our hands in virtual space
For a full-fledged interaction with the virtual environment, it is simply necessary that the VR device tracks the position of our hands, because it is difficult to come up with a better version of virtual manipulators.
There are many technologies to solve this problem, and Qualcomm has staked on cooperation with all specialized development companies, while giving preference to Leap Motion. The video below shows the implementation of hand interaction in a virtual space using the example of the training program for future cardiologists 'Think F.A.S.T.' Students are immersed in a virtual space with which they can interact using their own hands. Please note – there are no wires, sensors or antennas on the hands:
The main task that the developers solve is to minimize delays in the machine 'recognition' of objects as human hands.
Hocus pocus with focus
The human eye does not see the whole world in front of itself with the same quality. Closer to the periphery of vision, the picture is always processed with lower quality than in focus. Meanwhile, all VR glasses and helmets deliver the same image quality across the entire display area. Qualcomm has developed and implemented the 'Adreno Foveation' technology in the Snapdragon 845.
Its essence lies in the fact that the virtual reality device tracks the direction of the user's gaze and builds a picture in this place with the maximum detail of the object. This frees up computing resources and speeds up all rendering processes. And it makes the virtual world even more like the real one.
New destinations in XR
Voice interface will be important for the next generation of XR devices, especially for augmented reality (AR). AR glasses can have a voice control module that will run continuously, recognizing key phrases like 'Hey Snapdragon' to either wake up the glasses or trigger a specific action. Future AR glasses could use Snapdragon's computer vision and facial recognition technologies along with a trained neural network on the device to identify people in a room. With this information, AR glasses apps can either recommend actions or automatically take actions, such as displaying useful information about the people in the room.
AR could also transform shopping and mobile payments. For example, you can add items to your shopping cart simply by looking at them and tapping on the glasses (or poking your finger at them). When it comes time to pay, the glasses will identify you (scan your retina) and free up funds to pay.
Echoes in a tiny room and a sense of spaciousness
A full-fledged sound environment is a prerequisite for immersing yourself in virtual space. It has long been proven that the human ear perceives sound signals with a frequency of about 16 to 20,000 Hz (vibrations per second), as well as the fact that on the basis of the sound received, the brain is able to determine the dimensions of space, direction, etc. 100% fake surround sound is what you need to fully immerse yourself in virtual space.
Knowing this very well, Qualcomm's developers in the Snapdragon 845 have implemented full-fledged 3D sound, which undergoes special processing before playback, based on the conditions and characteristics of the virtual environment.
If you are standing in a virtual cave, you will hear the reverberations and reflections of your voice just like in a real cave.
All new Qualcomm chipsets sooner or later end up in top-end devices in related areas. For example, Snapdragon APQ8064 (600) was renamed 8065 and installed in expensive Wi-Fi routers, Snapdragon 835 settled in augmented reality glasses ODG R-8 and 9. The same fate awaits Snapdragon 845. The manufacturer offers a ready-made chipset, with many built-in devices and sophisticated software, and the client only needs to develop a design and hire programmers (who can use Qualcomm software tools) to create a proprietary interface.
How will this affect smartphone users with Snapdragon 845 on board? Will all these new Qualcomm virtual chips show up when launching standard VR apps from Google Play? Sooner or later these questions will be answered, but for now I propose to discuss more important questions. How accessible and massive virtual reality can change our world? Will wearing business VR glasses become a condition for adhering to the dress code in large companies? Who will be the first to enter the virtual world?