Sony's New AI Patent Predicts Button Presses via Finger-Camera Tech

Sony's latest patent filing hints at a significant leap forward in reducing latency in future gaming hardware, leveraging an AI model supported by additional sensors. The introduction of PlayStation Spectral Super Resolution (PSSR) with the PlayStation 5 Pro marked Sony's foray into upscaling technology, enhancing smaller resolutions to 4K. However, advanced graphics technologies such as frame generation can introduce latency, which might detract from the game's responsiveness.

According to the patent, latency between a user's input and the system's processing can lead to delayed command execution and unintended in-game consequences. Sony's proposed solution involves a multi-faceted approach: an AI model that predicts the next input and an external sensor, such as a camera monitoring the controller, to anticipate which button will be pressed next. The patent suggests that "the method may include providing camera input as an input to a machine learning (ML) model," with the camera input indicating the upcoming user command.

Additionally, the patent explores the use of controller buttons as sensors, aligning with Sony's history of innovation with analog buttons. While the exact implementation might differ from what's described in the patent, it's evident that Sony is committed to reducing latency without sacrificing responsiveness, particularly in light of popular rendering technologies like FSR 3 and DLSS 3, which introduce additional frame latency.

This technology could revolutionize gameplay in genres like twitch shooters, where high frame rates and low latency are crucial. However, it remains to be seen whether this patent will translate into actual hardware advancements.