The debate between general-purpose (GP) synthetic benchmarks and application-specific benchmarks (ASBM) for evaluating GPU performance lacks a definitive answer. Both types of benchmarks have their advantages and limitations. A GP synthetic benchmark is designed to evaluate a GPU across a wide range of applications, providing a broad performance overview. However, its workload may not fully capture the nuances of specific applications. Conversely, an ASBM is tailored to evaluate performance for a particular application, offering precise insights for that specific use case but lacking broader applicability.
If a general-purpose (GP) synthetic benchmark does a good enough job of evaluating a GPU for various applications, do we still need application-specific benchmarks (ASBM)?
There is no answer to this question, nor will there ever be one. Any benchmark is only as good as its workload. An ASBM will have a very limited workload, and so its results will not match those of a broad-based multi-workload benchmark. Which one is better? Well, if you’re only going to pay for one game, then the ASBM is better for you. But if you’re going to play multiple games, obviously, a properly designed synthetic is better.
There is also the issue of cheating. As has been demonstrated in the past, a GPU or AIB company can tweak its driver to give it a really great benchmark score. It’s more difficult with a synthetic benchmark, but doable, nonetheless.
As game developers add ray-tracing capability to their games (Nvidia claims over 500 games have adopted its AI-based DLSS ray-tracing feature), gamers desire a built-in benchmark. That is due in part to the lack of an easily accessible and respected ray-tracing benchmark. However, UL has a ray-tracing benchmark, and Basemark just introduced a cross-platform benchmark, so the need for a built-in one is seemingly moot.
The original idea for putting a benchmark in a game was to allow the potential buyer of the game to find out if his or her existing AIB or GPU could adequately run the game. But with the plethora of websites doing benchmarking, end users no longer have to test their gear; they can just find a site that’s done the work for them. And for games that don’t have a built-in benchmark, maybe there are sites, like our BabelTechReviews, that create a benchmark to run the evaluations.
So the answer is, benchmarks? Sure, you want ASBM, site-designed, or GP?
At Mt. Tiburon Testing Labs and BabelTechReviews, we use them all.