Arm has developed a temporal upscaling solution for mobile devices, building on AMD’s FidelityFX Super Resolution 2 technology. The approach combines information from multiple frames to produce high-quality images from lower-resolution targets, addressing graphical performance issues and providing benefits for mobile game developers. Unlike traditional spatial upscaling methods, Arm’s temporal upscaling solution offers improved image quality and reduced thermal throttling, leading to a better user experience. The company has demonstrated significant GPU performance improvements through testing on a commercial mobile device and is sharing its solution with the developer community through an MIT open-source license.
What do we think? Arm has been late to the scaling race, which, given all the turmoil with trying to sell it to Nvidia and not, then going public, and then being buried in Softbank, plus management departures and other shake-ups, you can understand things slipping. To the company’s credit, they chose existing technology, which saved considerable time and money. Arm is now in league with Apple’s Metal scaling, Imagination Technologies’ PowerVR super resolution, and Samsung’s Xclipse upscaler or neural upscaler technology (also from AMD).
Arm shares results of its ASR for upscaling on mobile
Arm recently published a blog about their new Arm Accuracy Super Resolution (Arm ASR) for upscaling on mobile devices.
Upscaling involves rendering certain stages of a frame at a lower resolution and subsequently applying a technique to scale the frame up to a higher resolution. It should not be used throughout the entire rendering process, as it can cause visual or user interface defects. However, when applied earlier in the pipeline, it can be highly effective, incorporating anti-aliasing as part of the process.
In the lighting-based post-processing stage, it manages screen-space reflections, screen-space ambient occlusion, denoisers (shadows, reflections), and exposure, and in the image-based post-processing stage, it does film grain, chromatic aberration, vignette, and tone mapping.
Arm said it explored various options and was impressed by AMD’s FidelityFX Super Resolution 2 (FSR 2), a component of the GPUOpen initiative, an open-source project launched in 2016. With its permissive MIT license, GPUOpen aims to offer versatile solutions that bridge the gap between PC and console hardware. Arm was drawn to these principles and began to consider what an ideal solution for mobile devices would entail.
Upscalers come in two types: spatial and temporal. Spatial upscaling, the first to be available to developers, works on a frame-by-frame basis. It is simpler to implement and requires less from the hosting game engine. Examples include FSR 1 and Qualcomm’s Game Super Resolution (GSR). However, this technique limits developers from rendering at too low a resolution to avoid blurred final images, though it is computationally inexpensive.
Temporal upscalers, on the other hand, are more complex. They combine information from multiple frames to produce the final image. This generally results in higher-quality images from lower-resolution targets but requires additional input from the engine, such as depth and motion vector information. A reactive mask is also desirable for handling features like particle effects that lack depth or motion information.
Arm opted to bypass traditional upscaling methods and instead developed a temporal upscaling solution to tackle prevalent graphical performance issues, providing advantages for mobile game developers. Leveraging AMD’s FSR 2 upscaler as a foundation, Arm created a modified solution that inherits the familiar API and configuration options, making it easy for developers to use. Although FSR 2 yields excellent results, its original implementation is only feasible on high-end PCs and consoles, prompting Arm to adapt the technology for mobile-specific needs.
Arm ran a series of tests and analyzed the results from a commercial mobile device that had an Arm Immortalis-G720 GPU and a display with 2800×1260 resolution. The company was able to demonstrate significant GPU performance improvements, as depicted in the following chart.
Rendering at a native resolution on a mobile device often leads to thermal throttling, which in games can ruin the user experience and shorten engagement. Arm’s technique, however, renders high-quality results at a stable, low temperature, the company claims. Arm says ASR delivers a performance boost of up to 1.5 times that of traditional FSR 2, while utilizing only one-third of the GPU resources. This results in frame rate increases of 20%–40%.
Arm says the enhanced performance also yields significant power efficiency gains, resulting in extended battery life—a crucial advantage in daily smartphone usage. In partnership with MediaTek, Arm substantiated that claim through testing on a Dimensity 9300-powered device, which yielded the following results.
Arm is pretty excited about the project’s outcome and eager to share it with the developer community through an MIT open-source license. By doing so, they are enabling developers to experience Arm’s ASR and explore its potential in their own projects. Anyone interested in being an early adopter should contact the company.