Apple Vision Pro includes a feature called EyeSight, which scans a user’s facial expressions and displays them on the headset’s outer screen, but it hasn’t received much positive reviews. However, a new patent suggests that Meta is looking to implement a similar feature in its upcoming VR devices. The patent, which can be viewed here, is titled “Sensors embedded in immersive reality headsets for enabling social presence.” The patent details how sensors inside a VR headset would read changes in a person’s face and use machine learning to correlate them with facial expressions.
But as far as I can tell, the patent doesn’t mention an external screen displaying a digital avatar like Vision Pro’s EyeSight. The patent only mentions facial movement data being delivered to an “immersive reality application.” It also mentions the ability to opt out of certain data sharing due to privacy concerns, and “correcting” the user’s facial expressions for the viewer in the immersive reality application.
Illustration showing the proposed sensor and facial interface. Image credit — Meta
At first glance, this looks like a copy of EyeSight, but I think it will be used for something else. There is no mention of an external screen, but the sheer number of references to immersive reality applications suggests that this will be used for digital avatars in VR games and apps. Take VRChat, the popular VR social hub, for example. There is a way to make gaze and face tracking work in VRChat if you are using a supported headset. If Meta implements what is described in this patent, users will not need such workarounds, and more VR platforms will be ready to support face-tracking digital avatars.
Of course, this is just a patent and doesn’t mean anything concrete at this point, but we wouldn’t be surprised if the Meta Quest 4 comes with advanced face tracking capabilities as outlined in the patent.
