The visionOS operating system that runs on Apple’s Vision Pro headset offers an unusual feature called “visual search”. The Visual Lookup tool offered on iPhones and iPads is similar to this feature. By using their headset and Visual Search, users can learn more about various objects, recognize and interact with text in their immediate environment, and even perform tasks such as copying and pasting printed text into the software.
Also read: Apple Embraces WebXR Standard with Vision Pro AR Headset for Browser-based VR
The Vision Pro headset’s Visual Search feature now includes text translation between 17 different languages. This technology gives users the ability to quickly translate text in real-time, which is particularly useful when traveling or in other circumstances where it is necessary to quickly interpret the text around them.
Users can interact with actual text and documents using the visual search tool that works within visionOS. This includes gathering contact information, clicking on links to other websites, converting measurements, and more. For example, users can quickly scan a printed brochure with the Apple Vision Pro headset and launch the website in the Safari browser if it has a web link. The headset can also easily convert units if a recipe calls for measurements in grams but the user prefers ounces. In addition, the headset’s automatic word and document recognition, similar to the iPhone’s ability to identify images, allows users to interact with such information quickly.
As Apple has just released the first developer preview of visionOS software, developers can use the latest Xcode beta to access the visual search feature. The integration of visual search into Apple’s immersive Vision Pro headset has taken a significant step forward.
The visionOS operating system’s visual search capability enables users to retrieve information, interact with text in their environment and perform a variety of tasks more easily. It is another development in Apple’s ongoing mission to enhance the user experience through the use of cutting-edge technologies.
