Recently, a member of the tech
press asked us about the status of AIXPRT,
our benchmark that measures machine learning inference performance. We want to
share our answer here in the blog for the benefit of other readers. The writer said
it seemed like we had not updated AIXPRT in a long time, and wondered whether we
had any immediate plans to do so.
true that we haven’t updated AIXPRT in quite some time. Unfortunately, while a
few tech press publications and OEM labs began experimenting with AIXPRT
testing, the benchmark never got the traction we hoped for, and we’ve decided
to invest our resources elsewhere for the time being. The AIXPRT installation
packages are still available for people to use or reference as they wish, but
we have not updated the benchmark to work with the latest platform versions
(OpenVINO, TensorFlow, etc.). It’s likely that several components in each
package are out of date.
are interested in AIXPRT and would like us to bring it up to date, please let us know.
We can’t promise that we’ll revive the benchmark, but your feedback could be a
valuable contribution as we try to gauge the benchmarking community’s interest.
One of the core principles of
the BenchmarkXPRT Development Community is a commitment to valuing the feedback
of both community members and the larger group of testers that use the XPRTs on
a regular basis. That feedback helps us to ensure that as the XPRTs continue to
grow and evolve, the resources that we offer will continue to meet the needs of
those that use them.
In the past, user feedback has influenced specific aspects of our benchmarks such as the length of test runs, user interface features, results presentation, and the removal or inclusion of specific workloads. More broadly, we have also received suggestions for entirely new XPRTs and ways we might target emerging technologies or industry use cases.
approach the second half of 2022 and begin planning for 2023, we’re asking to
hear your ideas about new XPRTs—or new features for existing
XPRTs. Are you aware of hardware form factors, software platforms, or prominent
applications that are difficult or impossible to evaluate using existing performance
benchmarks? Are there new technologies we should be incorporating into existing
XPRTs via new workloads? Can you recommend ways to improve any of the XPRTs or
XPRT-related tools such as results viewers?
We are interested in your answers to these questions and any other ideas you have, so please feel free to contact us. We look forward to hearing your thoughts!
new to the XPRT benchmarks may not know about one of the free resources we
offer. The XPRT results database currently holds more than 3,000 test results
from over 120 sources, including major tech review publications around the
world, OEMs, and independent testers. It offers a wealth of current and
historical performance data across all the XPRT benchmarks and hundreds of
We update the results
database several times a week, adding selected results from our own internal
lab testing, reliable tech media sources, and end-of-test user submissions.
(After you run one of the XPRTs, you can choose to submit the results, but they
don’t automatically appear in the database.) Before adding a result, we
evaluate whether the score makes sense and is consistent with general
expectations, which we can do only when we have sufficient system information details.
For that reason, we ask testers to disclose as much hardware and software
information as possible when publishing or submitting a result.
We encourage visitors to our site to explore the XPRT results database. There are three primary ways to do so. The first is by visiting the main BenchmarkXPRT results browser, which displays results entries for all of the XPRT benchmarks in chronological order (see the screenshot below). You can narrow the results by selecting a benchmark from the drop-down menu and can type values, such as vendor or the name of a tech publication, into the free-form filter field. For results we’ve produced in our lab, clicking “PT” in the Source column takes you to a page with additional disclosure information for the test system. For sources outside our lab, clicking the source name takes you to the original article or review that contains the result.
The second way to access our published results is by visiting the results page for an individual XPRT benchmark. Go the page of the benchmark that interests you, and look for the blue View Results button. Clicking it takes you to a page that displays results for only that benchmark. You can use the free-form filter on the page to filter those results, and can use the Benchmarks drop-down menu to jump to the other individual XPRT results pages.
The third way to view
information in our results database is with the WebXPRT 4 results viewer.
The viewer provides an information-packed, interactive environment in which
users can explore data from the curated set of WebXPRT 4 results we’ve
published on our site. To learn more about the viewer’s capabilities and
features, check out this blog post
We hope you’ll take
some time to browse the information in our results database. We welcome your feedback
about what you’d like to see in the future and suggestions for improvement. Our
database contains the XPRT scores that we’ve gathered, but we publish them as a
resource for you. Let us know
what you think!
Back in March, we discussed
the WebXPRT 4 results submission process and reminded readers that everyone who
runs a WebXPRT 4
test is welcome to submit scores for us to consider for publication in the WebXPRT 4 results viewer.
Unlike sites that publish every result that users submit, we publish only
results that meet our evaluation criteria. Among other things, scores must be
consistent with general expectations and must include enough detailed system
information to help us assess whether individual scores represent valid test
runs. Today, we offer a couple of tips to increase the likelihood that we will
publish your WebXPRT 4 test results.
Tip 1: Specify your system’s processor
While testers usually include
detailed information for the device, model number, operating system, and
browser version fields, we receive many submissions with little to no information
about the test system’s processor.
In the picture below, you can see an example of the level of detail that we require to consider a submission. We need the full processor name, including the manufacturer and model number (e.g., Intel Core i9-9980HK, AMD Ryzen 3 1300X, or Apple M1 Max). Note that we do not require the processor speed reported by the system.
Tip 2: Include a valid email
It is also common for submissions
to not include a valid email address. While we understand the privacy concerns related
to submitting a personal or corporate email address, we need a valid address
that we can use as a point of contact to confirm test-related information when
necessary. We don’t use those addresses for any other purposes, such as selling
them, sharing them with any third parties, or adding them to a mailing list.
We hope this information explains why we might not have published your results. We look forward to receiving your future score submissions. If you have any questions about the submission process, please let us know!
Many of our blog
readers first encountered the XPRTs when reading about a specific benchmark,
such as WebXPRT, in a device
review. Because these folks might be unfamiliar with our other benchmarks, we
like to occasionally “reintroduce” individual XPRTs. This week, we invite you
to get to know HDXPRT.
stands for High-Definition Experience & Performance Ratings Test, was the
first benchmark published by the HDXPRT Development Community, which later
became the BenchmarkXPRT Development Community. HDXPRT 4, the latest version, evaluates
the performance of Windows 10 and Windows 11 devices while handling real-world
media tasks such as photo editing, video conversion, and music editing. HDXPRT
uses real commercial applications, such Photoshop and MediaEspresso, to
complete its workloads. The benchmark then produces easy-to-understand results
that are relevant to buyers shopping for new Windows systems.
The HDXPRT 4
setup process takes about 30 minutes on most systems. The length of the test
can vary significantly depending on the speed of the system, but for most PCs that
are less than a few years old, a full three-iteration test cycle takes under two
HDXPRT is a useful
tool for anyone who wants to evaluate the real-world, content-creation
capabilities of a Windows PC. To see test scores from a variety of Windows
devices, go to HDXPRT.com and
click View Results.
Want to run HDXPRT?
Download HDXPRT from
HDXPRT.com. The HDXPRT user manual provides information
on minimum system requirements, as well as step-by-step instructions for
configuring your system and kicking off a test.
HDXPRT 4, our benchmark for assessing Windows performance on real-world media tasks, runs tests that use real commercial applications such as Adobe Photoshop Elements (PSE) 2020. Last fall, we informed HDXPRT testers that Adobe had started requiring a user ID to download the free Adobe Photoshop Elements 2020 trial package. Previously, testers could download the trial without setting up an account.
Recently, Adobe made additional changes to the access path for the
PSE 2020 installation package. The package is no longer available on the PSE downloads page, but users who
previously purchased their copy or registered it with Adobe can access the
package on another page. However, this approach does not work for users who want to
temporarily use the trial version for HDXPRT 4 testing.
We have found a third-party location, ProDesignTools, that
currently offers a free, straightforward PSE 2020 installation package download
with no requirements for registration or transmission of personal information.
In our testing so far, the installation package (PhotoshopElements_2020_LS30_win64_ESD.zip)
has been functioning as expected, and HDXPRT 4 is running the PSE-based
workloads without any issues.
Unfortunately, we cannot guarantee that ProDesignTools will continue to offer a free PSE 2020 installation package download, and we’re not aware of an alternative Adobe download path at this time. We apologize for the inconvenience!