The Standard Performance Evaluation Corporation’s Graphics and Workstation Performance Group (SPEC/GWPG) provides updates on their activities around the Siggraph time frame, focusing on three primary committees: SPECapc, SPECgpc, and SPECwpc. Their goals include developing industry-standard workstation and graphics performance benchmarks, increasing downloads, collaborating with software developers, and recruiting members. Recently, SPECapc released benchmarks for Creo 9 and Maya 2023/2024, with updates and improved test cases. Upcoming releases include SPECapc for 3ds Max 2024 and SolidWorks 2024. SPECgpc plans a major revision of SPECviewperf 15, incorporating new APIs and technologies. SPECwpc 4 will bring significant improvements, particularly in AI, with an overhaul of workloads and the GUI, category changes, and automation via CLI. SPEC encourages industry collaboration to develop robust benchmarks based on commercial applications, offering free downloads for users and licensing fees for vendors.
What do we think? Where would we be without industry-standard benchmarks, especially those that use actual applications? It’s not easy, having developed an in-application benchmark back in the 1990s, I speak from experience. SPEC has the talent and the history, and has become a trusted source for reliable and objective testing, and, therefore, comparison of the hardware platforms under test.
The latest from SPEC/GWPG
The Graphics and Workstation Performance Group (GWPG) of the Standard Performance Evaluation Corporation (SPEC) provides an update on their activities once or twice a year, but always around the Siggraph time frame—or at least they try to, though sometimes developments take longer than anticipated.
The SPEC/GWPG has three primary committees:
- SPECapc—Develops application-based benchmarks that measure total system performance. Offers five benchmarks based on CAD/CAM and media/entertainment applications.
- SPECgpc—Develops SPECviewperf, the world’s most widely used benchmark for computer systems running graphics-intensive professional applications.
- SPECwpc—Develops SPECworkstation, a comprehensive benchmark that measures key aspects of workstation performance based on diverse professional applications.
The organization’s goals are pretty straightforward (and have been since 1988):
- Continue to develop industry-standard workstation and graphics performance benchmarks for today’s most popular workstation applications.
- Obtain new, modern models, whether donated or commissioned.
- Continue to increase GWPG downloads.
- Increase technical and marketing collaboration with ISVs that develop applications on which benchmarks are based.
- And, recruit new members.
So, what have you done for me lately?
SPECapc has been continuing to develop industry-standard workstation and graphics performance benchmarks for today’s most popular workstation applications. To do that, they:
- Obtain new, modern models, whether donated or commissioned.
- Continue to increase GWPG downloads.
- Increase technical and marketing collaboration with ISVs that develop applications on which benchmarks are based.
- Recruit new members.
For example, in July, they released SPECapc for Creo 9. It provides eight workflows (three CPU/five GPU) that exercise all aspects of system performance when running Creo 9. The benchmark updates include:
- Updated models to support Creo 9, including a new submarine model and an updated world car model.
- Updated CPU test cases, including clipping and mass properties tests and a new CPU test case for render studio.
- Updated GPU test cases, including model rotations in different graphic views, exploded and cross-section views, with complex lighting, background, and more.
They’ve also recently released SPECapc for Maya 2023 and 2024. The 2024 benchmark consists of 43 tests using 11 different models and animations. It includes eight different graphics tests in various modes and five different CPU tests. The graphics-oriented tests use six different Maya view settings—Shaded, Shaded SSAO, wireframe on shading, wireframe on shaded SSAO, textured, and textured SSAO. There are various tests that measure both animation and 3D model rotation performance. Five CPU tests within the benchmark perform CPU ray tracing and evaluation caching in various modes.
The new benchmark includes the SPECapc for Maya 2024 Workload Development Kit, a framework allowing users to run their own GPU-specific workloads using the same measurement and output techniques.
Coming up in Q4 2023 and into 2024 are SPECapc 3ds Max 2024 with a new benchmark, not just a port forward. SPEC has been collaborating with Autodesk for benchmark feedback and developing partnerships with 3D content creators.
There will also be a SPECapc for SolidWorks 2024. It will add new modes to the benchmark, and the SPEC folks say the development is almost complete and is targeted for release to coincide with the application release.
The popular SPECgpc, SPECviewperf 15 will bring a new major revision. They plan to include new APIs (DX12, Vulkan) and technologies (path tracing) based on actual application usage. There will also be further enhancements to installation experience and better support of display DPI scaling.
Not to be left out, SPECwpc has some major improvements coming, especially in AI, with SPECworkstation 4, with:
- Overhaul of workloads and GUI.
- Vertical category changes.
- New ML/AI vertical.
- General Operations -> Productivity and Development.
- Product Development -> Product Design.
- Subsystem category changes (GPUCompute -> Accelerators).
- Targeting reduced runtime.
- Minimal tuning parameters (e.g., no more “threads per process”).
- Automation via CLI.
SPEC, somewhat like what the folks at Khronos do, encourages collaboration among the industry’s premier developers to build robust benchmarks measuring the most relevant use cases for popular workstation applications. Benchmarks are based primarily on commercial applications. They have a free download policy, which encourages widespread distribution of benchmarks to users, while vendors of computer products pay for the right to use benchmarks—$2,500 for SPECviewperf and SPECapc, and $5,000 for SPECworkstation. The organization has kept a small team, with all members contributing to benchmark development. There is also an investment in comprehensive public relations and marketing aimed at informing target audiences, generating downloads, and engaging editors and publishers. The goal has always been to have shorter, intense development cycles with two or more new or updated benchmarks released each year.