The recent advancements in Meta’s technology have brought virtual reality (VR) closer to simulating the real world. The company’s focus on creating a more realistic and immersive metaverse has led to the development of new features that allow virtual objects to have real shadows and highlights. Last year Meta launched the mixed reality development toolkit MR Utility Kit at the Connect 2023 conference.
This is a toolkit specially designed for mixed reality application developers, which contains various practical functions, prefabricated components and design guidelines. The toolkit is now available for Unity and Unreal and offers an exciting new feature called Passthrough Relighting. This feature allows developers to easily add realistic shadows and highlights to digital objects in mixed reality. This significantly improves the immersion and visual effects of mixed reality.
Shadows and highlights
Historically, simulating realistic shadows and highlights in mixed reality has been a technically complex process that required computer graphics expertise. Passthrough Relighting has completely changed this situation. It provides reusable components, sample code, detailed documentation and design guidelines, which greatly reduces the difficulty of development. This feature also allows developers to easily match the light of the virtual world with the real environment. It seamlessly integrates digital objects with the real environment.
Meta hopes to make mixed reality more realistic and attractive through Passthrough Relighting. The above image visually shows three different effects: the left side is the original state without any shadows, the middle is the effect after adding simple shadows, and the right side is the realistic shadow effect brought by Passthrough Relighting. The image below shows how highlights are reflected from the real environment onto virtual objects.
Currently, developers need to integrate these effects into their mixed reality apps for users to experience their power. So it might be a while before we see these effects in action in apps. Mixed reality sandbox game Figmin XR is expected to be one of the first apps to benefit from Passthrough Relighting technology. Developer Javier Davalos has hinted that this feature will be integrated into the upcoming version 21 update.
Passthrough relighting, allows users to see the real world while wearing their VR headsets. Meta’s passthrough technology has been the subject of both praise and criticism, with ongoing efforts to improve its quality.
Current State of Passthrough Technology
Meta’s passthrough technology has been met with mixed reviews. The Meta Quest Pro (MQP) has been criticized for its low-resolution passthrough mode, which hinders its effectiveness for augmented reality (AR) applications. The passthrough mode on the Quest 3 has also been noted to suffer from issues such as grainy images in low light and distortion around objects, indicating the need for further improvement. However, Meta is still working on the technology and the company is now making good progress.
Gizchina News of the week
Meta has been working on prototypes that showcase advancements in passthrough technology. These include the Flamera and Butterscotch prototypes, which demonstrate progress in light field passthrough and retinal resolution, respectively. The Flamera prototype, in particular, has been lauded for its high-quality AR passthrough, with a focus on providing a more immersive and accurate representation of the real world.
Future Prospects
Meta has expressed its commitment to improving passthrough technology over time through software updates. The company’s Chief Technology Officer, Andrew Bosworth, has confirmed that the passthrough quality on the Quest 3 will continue to be enhanced, addressing issues such as image distortion and graininess. These efforts reflect Meta’s ongoing dedication to enhancing the user experience and pushing the boundaries of VR and AR technology.
Passthrough relighting technology has a promising future, with Meta’s ongoing research and development efforts aimed at improving the quality of passthrough experiences. Meta has also been working on prototypes that showcase advancements in passthrough technology, such as the Flamera prototypes. The Flamera prototype, in particular, has been lauded for its high-quality AR passthrough. It focuses on providing a more immersive and accurate representation of the real world.
Meta’s commitment to improving passthrough technology over time through software updates and hardware advancements reflects the company’s dedication to enhancing the user experience and pushing the boundaries of VR and AR technology. As Meta continues to receive real-world lighting conditions and information from headsets, the algorithms that drive passthrough technology will be tuned more effectively. This will lead to more stable and high-quality passthrough experiences.
Final Words
Though difficult to achieve, realistic shadows and highlights can now be used with Meta’s Passthrough Relighting technology. This feature allows developers to easily match the light of the virtual world with the real environment. It seamlessly integrates digital objects with the real environment. However, Meta’s passthrough relighting technology has seen both challenges and advancements. While current iterations have been criticized for their limitations, the company’s ongoing research and development efforts are evident. This can be seen in recent prototypes like Flamera and Butterscotch. Such prototypes signal a promising future for more immersive and high-fidelity passthrough experiences. With a focus on innovation and user feedback, Meta is also poised to make significant strides in passthrough technology.
Author Bio
Efe Udin is a seasoned tech writer with over seven years of experience. He covers a wide range of topics in the tech industry from industry politics to mobile phone performance. From mobile phones to tablets, Efe has also kept a keen eye on the latest advancements and trends. He provides insightful analysis and reviews to inform and educate readers. Efe is very passionate about tech and covers interesting stories as well as offers solutions where possible.
