Volumetric video is a technology that captures performances from multiple cameras, generating a 3D representation of that performance that can be later viewed from any angle at any point in time. This is a new way of experiencing immersive content, not only because of the new visualization possibilities but also because of its true-to-life characteristics. With volumetric video, reality can be transported into the virtual world.
But to make it happen, a lot of data needs to be well-managed and transferred along the creative pipeline. Volumetric video is basically a sequence of 3D models captured frame by frame, and for each frame, it requires the storage of geometric information (the shape of what is being captured) and texture (the color and appearance of the geometry surface), resulting in a huge amount of data. A powerful compression system is essential to work with this technology, here is why:
Freedom To Choose Your Favorite Capture Solution
Choosing the best capture solution for the creation of your experience is one of the most important steps of your project. Capture solutions differ in visual quality, pricing, file sizes, and many other characteristics, so choose wisely but also choose freely. Remember that you don’t have to capture all elements of your immersive project with the same capture solution, as you might want to have better visual quality for your main character, for example, while being comfortable having background characters with lower quality and possibly better pricing.
The issue with freely mixing holograms of different capture solutions is that most editing and publishing tools do not support and/or cannot compress different file formats. With Sense XR, you can either use OBJ and Alembic sequences (given by most capture solutions as an open format) or use proprietary formats (we currently support 4DViews’ .4DS, TetaVi’s .TET, Mantisvision’s .MVX, and Microsoft’s .MP4.) to create your experience with as many capture solutions as you want.
One of the most important features in Sense XR is to offer the best compression system for smoothly streaming your holograms on the Web and WebAR, regardless of your capture technology choice and allowing you to mix different capture technologies. Our compression system is automated and works entirely on the cloud, so you don’t need to put any effort to make it happen, you just need to push a button.
Higher Visual Quality Depends On Good Data And Good Compression
If you have chosen to work with volumetric video holograms, you probably understand the value of photographic realism in immersive content creation. But while you are looking for the capture solution that offers the best visual quality, you should also remember that the more details you have in a capture, the higher the size of the file, so you should keep in mind how your data will be compressed during your editing process.
The balance game here is to not compromise the great visual quality of your experience while also keeping file sizes as small as possible. With Sense XR’s compression system, we use a combination of geometry compression and texture compression, as well as reducing invisible details of the geometry. This way, we can ensure good visual quality without jeopardizing the performance and responsiveness of the experience.
Creatives should also know that there are other major factors affecting the visual quality of a hologram after compression, and these factors are related to how the capture solution prepares their data and delivers it to their clients. Normally, high end capture solutions such as Microsoft and Mantis Vision deliver their data in such an organized way that after compression they can reach a considerably better visual quality for less data.
Fluent Streaming Of Holograms Demands Efficient Compression
The performance and responsiveness of your experience are greatly impacted by your compression system. With smaller files, your viewers will be able to experience your holograms in a more fluid manner, as less data is being transferred and managed by their network and viewing device. This is especially important when streaming Web and WebAR experiences, meaning that no data has been previously stored on the viewer’s device through an app download.
Traditionally, holographic experiences have required the user to download and preprocess an entire volumetric video before showing the experience. With the introduction of fast and effective decompression systems, volumetric videos can now be streamed. This means that the playback can be started as soon as a very small portion of the volumetric video is downloaded, and the experience is downloaded and decompressed in real time while it plays. By supporting adaptive streaming, the volumetric video quality can be changed in real time to respond to changes or disturbances in the network connection, thus ensuring a smooth and undisturbed experience.
Hologram data from Volumetric Wonderland by zyntroPICS, captured at DGene Studio – LA.
To sum up, compression is an essential subject when creating immersive experiences with heavy data. By choosing editing and publishing solutions that have efficient compression systems that don’t compromise visual quality, you will also have in hand the freedom to choose your favorite capture solutions, high visual fidelity, and the fluid streaming of your creation.
With Sense XR you can create your hologram scene with any capture technology choice and publish it to WebAR with a single click. Your holograms will be magically compressed in the best format for the web and you’ll instantly get a web link and a QR code with your holograms running in the only WebAR with native adaptive streaming for volumetric videos in the world. Ready to share with anyone.
